Tuesday, 2018-11-06

*** ansmith has joined #openstack-oslo00:00
*** phuongnh has joined #openstack-oslo00:44
*** ducnv has quit IRC01:18
*** zaneb has joined #openstack-oslo01:31
*** zaneb has quit IRC01:31
*** zaneb has joined #openstack-oslo01:31
*** phuongnh has quit IRC01:57
*** phuongnh has joined #openstack-oslo01:58
*** ianychoi has quit IRC02:10
*** ianychoi has joined #openstack-oslo02:10
*** toabctl has quit IRC02:19
*** toabctl has joined #openstack-oslo02:23
*** ducnv has joined #openstack-oslo02:41
*** lbragstad has quit IRC03:16
*** cfriesen has quit IRC03:27
*** cburgess has quit IRC03:41
*** cburgess has joined #openstack-oslo03:41
*** dave-mccowan has joined #openstack-oslo04:28
*** dave-mccowan has quit IRC04:53
*** prometheanfire has joined #openstack-oslo05:58
prometheanfirehttps://review.openstack.org/615724 fixes a nova thing, more reviews would be nice (oslo.service)05:58
prometheanfire:D05:58
*** e0ne has joined #openstack-oslo06:24
*** e0ne has quit IRC06:44
*** e0ne has joined #openstack-oslo06:48
*** phuongnh has quit IRC06:51
*** phuongnh has joined #openstack-oslo06:52
*** e0ne has quit IRC07:07
*** pcaruana has joined #openstack-oslo07:36
*** cfriesen has joined #openstack-oslo07:39
*** cfriesen has quit IRC07:48
openstackgerritwangxiyuan proposed openstack/oslo.limit master: Add basic enforce func  https://review.openstack.org/59652008:50
*** mattgo has joined #openstack-oslo09:05
*** mattgo has quit IRC09:05
ttxbnemec: it's a bit borderline... The cases where that would affect users are probably close to 0... so I could go either way09:15
ttx(x or y)09:15
ttxBut I would definitely not include that one in stable branches :)09:16
*** cdent has joined #openstack-oslo09:22
*** lbragstad has joined #openstack-oslo10:00
*** phuongnh has quit IRC10:42
openstackgerritwanghui proposed openstack/taskflow master: Remove stestr from requirements.txt  https://review.openstack.org/61582510:58
openstackgerritwanghui proposed openstack/oslotest master: Remove stestr from requirements.txt  https://review.openstack.org/61582611:01
*** lbragstad has quit IRC11:02
*** lbragstad has joined #openstack-oslo11:04
*** e0ne has joined #openstack-oslo11:11
*** raildo has joined #openstack-oslo11:54
*** pbourke has quit IRC12:11
*** pbourke has joined #openstack-oslo12:11
*** e0ne has quit IRC12:29
*** dave-mccowan has joined #openstack-oslo12:29
*** ansmith has quit IRC12:39
*** e0ne has joined #openstack-oslo12:43
*** cdent has quit IRC13:31
*** kgiusti has joined #openstack-oslo13:37
openstackgerritHervé Beraud proposed openstack/oslo.service stable/rocky: Fix stop of loopingcall  https://review.openstack.org/61526314:00
openstackgerritHervé Beraud proposed openstack/oslo.service stable/rocky: Use eventlet Event for loopingcall events  https://review.openstack.org/61448914:03
*** ansmith has joined #openstack-oslo14:07
*** bobh has joined #openstack-oslo14:19
openstackgerritHervé Beraud proposed openstack/oslo.service stable/rocky: Fix stop of loopingcall  https://review.openstack.org/61526314:25
openstackgerritMerged openstack/oslo.log master: Add Windows Event Log handler  https://review.openstack.org/61455814:29
openstackgerritHervé Beraud proposed openstack/oslo.service stable/rocky: Use eventlet Event for loopingcall events  https://review.openstack.org/61448914:30
*** munimeha1 has joined #openstack-oslo14:58
*** cfriesen has joined #openstack-oslo15:04
*** e0ne has quit IRC15:06
*** prometheanfire has left #openstack-oslo15:09
*** cdent has joined #openstack-oslo15:36
openstackgerritStephen Finucane proposed openstack/oslo.upgradecheck master: doc: Remove crud from conf.py file  https://review.openstack.org/60932716:26
openstackgerritStephen Finucane proposed openstack/oslo.upgradecheck master: doc: Switch to openstackdocstheme  https://review.openstack.org/60932816:26
dhellmannkgiusti : what is our current recommended driver for notifications? rabbit or kafka or something else?17:02
kgiustidhellmann: rabbitmq17:02
dhellmannI'm considering writing something that consumes notifications, but want to use asyncio so I don't think I'll be able to use oslo.messaging directly (at least in the early stages; I would consider adding asyncio support at some point). Do you think if I went with rabbitmq, that would make it work with most deployments?17:03
kgiustidhellmann: yes, definitely rabbitmq for that17:04
dhellmannk17:04
dhellmannkgiusti : thanks!17:04
kgiustidhellmann: yw17:05
openstackgerritHervé Beraud proposed openstack/oslo.config master: Introduce a base yaml parser for all openstack components  https://review.openstack.org/61552917:18
openstackgerritHervé Beraud proposed openstack/oslo.config master: Introduce a base yaml parser for all openstack components  https://review.openstack.org/61552917:20
*** e0ne has joined #openstack-oslo17:32
openstackgerritMerged openstack/oslo.service stable/rocky: Fix stop of loopingcall  https://review.openstack.org/61526317:34
*** e0ne has quit IRC18:03
bnemeckmalloc: zzzeek: Any thoughts on how to fix https://bugs.launchpad.net/oslo.cache/+bug/1801967 ?18:03
openstackLaunchpad bug 1801967 in oslo.cache "TypeError: unhashable type: 'list' with redis driver" [Critical,Triaged]18:03
kmallocbnemec: ugh.18:04
kmalloci think we're going to need to add a "handle arguments" per backend18:04
bnemecYeah18:05
kmallocthe core of the issue is the backends are opaque18:05
kmallocso we can't know unless we know what library is being used behind the scenes18:05
kmallocthis is an issue with dogpile not being consistent in it's backends18:05
kmallocand just letting the underlying library handle it18:05
kmalloci'll just re-write the memcache backend to be pymemcache and handle the url arguments18:06
kmalloclets revert my fix18:06
kmalloci'll overload the backend bits in oslo.cache for now and change the default behaviors there18:06
openstackgerritMorgan Fainberg proposed openstack/oslo.cache master: Revert "Config dict arguments.url must be a list"  https://review.openstack.org/61596018:06
*** e0ne has joined #openstack-oslo18:06
openstackgerritMorgan Fainberg proposed openstack/oslo.cache stable/rocky: Revert "Config dict arguments.url must be a list"  https://review.openstack.org/61596118:06
kmalloc^18:07
kmalloci should have a new driver written today to fix the core of that issue.18:07
bnemecSounds good, thanks18:07
kmallocbut it's much easier to just revert than chase redis fixes down18:08
kmallocalso... someone should chat with ceilometer and tell them they should never use redis unless they're using pre-new license.18:08
* kmalloc gets off soapbox on that last one.18:09
*** hberaud has quit IRC18:14
openstackgerritMorgan Fainberg proposed openstack/oslo.cache master: Revert "Config dict arguments.url must be a list"  https://review.openstack.org/61596018:14
openstackgerritMorgan Fainberg proposed openstack/oslo.cache stable/rocky: Revert "Config dict arguments.url must be a list"  https://review.openstack.org/61596118:15
*** hberaud has joined #openstack-oslo18:15
kmallocbnemec: ^ with proper bug tag18:15
*** e0ne has quit IRC18:24
*** efried has joined #openstack-oslo18:40
efriedmelwitt, bnemec, dhellmann: So as not to duplicate effort, I'm working on that fixture (https://review.openstack.org/#/c/615724/)18:41
melwittefried: woot, wfm18:42
*** prometheanfire has joined #openstack-oslo18:49
prometheanfiredoes osprofiler have an irc channel (or possibly share it here)?18:50
kmallocprometheanfire: afaik it's pretty dead.18:51
kmallocprometheanfire: in all of rocky, osprofiler had 35 reviews in the repo18:51
prometheanfireya18:51
prometheanfirekinda thinking that too18:52
kmallocprometheanfire: i'm about to rip it out of keystone18:52
prometheanfireI want them to remove jaeger-client from it18:52
prometheanfirethey aren't even using it18:52
prometheanfirenvm, git pull, they are using it18:52
prometheanfire:(18:53
*** bobh has quit IRC18:55
*** shardy has quit IRC19:00
kmallocbnemec: hm. so...19:12
kmallocbnemec: i need to handle serialization in the new memcache backend.... python-memcached uses pickle19:13
kmallocbut to keep the most consistent behavior, that is the best case.19:13
kmallocmaybe msgpack?19:13
bnemecI know we added msgpack to something else recently.19:14
bnemecMaybe messaging?19:14
kmallocoslo_serialization uses it19:14
kmallocbut it is not as clean as pickle19:14
kmallocpickle (sadly) just works19:14
kmallocmsgpack requires some special magic to load some classes back from serialization19:15
kmallocwe found this out in keystone (we use it there)19:15
kmalloci am inclined to just use pickle for now for consistency and then we do a deprecation cycle on that behavior19:15
kmalloc?19:15
openstackgerritEric Fried proposed openstack/oslo.service master: Fixture to mock loopingcall wait()  https://review.openstack.org/61597819:15
kmalloc... i guess i could just re-implement the same backend for now and do pymemcache separately19:16
efriedmelwitt, bnemec, dhellmann: ---^19:16
kmallocit's a 1-line fix.19:16
bnemeckmalloc: I guess I'm not that familiar with the architecture of oslo.cache so I'm not sure I can have an informed opinion on this.19:16
* kmalloc wrote most of the core of oslo.cache for keystone :P19:16
kmallocso, what happens is there are three tiers of objects: region, proxy, backend19:17
kmallocregion is the main interface, proxy catches data passed to the backend and modifies it19:17
kmallocand backends are like redis, pylibmc, python-memcached19:17
kmallocetc19:17
kmallocand implement to the lowest common/most basic functionality (python-memcached)19:18
kmallocthe issue is python-memcached doesn't understand "host:port, host:port, host:port" and expects [host:port, host:port, host:port]19:18
*** prometheanfire has left #openstack-oslo19:18
kmalloceverything else is fine with strings (As proofed by redis errors)19:18
bnemecRight19:19
kmallocalso python-memcached is effectively abandonware19:19
kmalloc1 year since the last merge of code19:19
kmallocand it is ... horribly designed.19:19
dhellmannefried : that looks reasonable to me. did you test it in the nova tests locally?19:19
bnemecHmm, so pylibmc is the preferred way to talk to memcached?19:19
kmallocand is explciitly thread-local focused19:19
kmallocbnemec: i would use pymemcached19:19
kmallocpymemcache*19:19
kmallocbut dogpile doesn't have that backend yet19:20
efrieddhellmann: no, left a note asking melwitt to do that, but I can if she doesn't wanna.19:20
kmallocso, maybe the solution is 1) fix python-memcached backend (in oslo.cache)19:20
kmallocjust a very basic fix.19:20
kmallocand 2) write a backend that uses pymemcache19:20
kmallocwhich is *very* good19:20
dhellmannefried : Ok. I think we want that class to show up somewhere in the docs. I don't remember if oslo.service uses the auto-doc stuff or not19:21
kmalloci was going to do it all at once, but this seems like i have other issues such as serialization/deserialization19:21
kmallocsince i *dont* want to use pickle19:21
dhellmannefried : it looks like it uses it partially19:21
kmallocunfortunately, complex objects seem to have issues with msgpack sometimes.19:21
efrieddhellmann: Roger that. It's mostly PoC at this stage. LMK what needs to happen to make it goable.19:21
dhellmannefried : I just left a note on the review19:22
dhellmannthanks for working on this!19:22
efrieddhellmann: ack, thanks19:22
kmallocalso, loading unverified data via pickle... can expose security issues19:22
kmalloce.g. if the memcache server is compromised... the runtime code could be changed19:22
kmallocmsgpack is a bit better on that front.19:22
dhellmannefried, bnemec : should we do a reno for the fixture, too? or are we keeping those for production users?19:22
bnemecYeah, I thought that was the main reason pickle is discouraged.19:22
*** cdent has quit IRC19:23
kmallocbleh... this is icky.19:23
kmallocso 1) i'll monkeypatch a fix for oslo.cache for python-memcache19:23
bnemecdhellmann: I think in the past you had said reno should be for user-facing things. But oslo is kind of weird in that in some cases our users are developers.19:23
kmalloci'll make a new backend that isn't crummy and isn't using pickle19:23
dhellmannbnemec : yeah, I always means "production user" in that context, but I don't know if that's how others interpret it19:24
kmallocand we can do a deprecation cycle where oslo.cache stops supporting python-memcache19:24
kmallocand we allow folks to register the serializer/deserializer mechanisms/specs for msgpack where it breaks19:24
kmallocmostly, in keysotne we have some wonky almost self-referential objects19:25
kmallocwhich is bad in itself.19:25
bnemecdhellmann: Maybe we should add a developer notes section to reno?19:25
bnemeckmalloc: Yeah, that sounds like something we should do.19:25
kmallocbnemec: hm i actually might be able to fix without a revert now that i think about it19:26
melwittefried: how to try it locally? patch it into oslo.service in my unit test venv?19:26
kmallocbnemec: if i have a fix that works sans revert, we'll roll with that before pushing the revert through19:26
efriedmelwitt: Yes, that should work. I usually git clone the project under .tox/pyXX/src and pip install -e it from within the venv; then I can do things like git review -d XXXXX to pull down different patch sets or whatever.19:27
bnemeckmalloc: Works for me. I already submitted the patch to block this release in g-r so as long as we don't release again nobody should be blocked.19:27
melwittefried: kewl, thanks. I'm a noob on that19:28
efriedmelwitt: lmk if you need help19:28
kmalloc++19:28
kmallocbnemec: perfect19:28
* bnemec needs to go vote and eat19:30
openstackgerritMorgan Fainberg proposed openstack/oslo.cache master: Only split url arg with memcache backend  https://review.openstack.org/61597919:31
kmallocbnemec: ^ that should be the "real" fix.19:32
kmallocooh or whoops, wont fix pool.19:33
efrieddhellmann: Save me digging, since I reckon you'll know this one off the top: What's the sphinx/rst form for linking to another doc in the same tree?19:40
openstackgerritMorgan Fainberg proposed openstack/oslo.cache master: Only split url arg with memcache backend  https://review.openstack.org/61597919:41
efrieddhellmann: nm, figgered it out.19:42
openstackgerritEric Fried proposed openstack/oslo.service master: Fixture to mock loopingcall wait()  https://review.openstack.org/61597819:46
jrollhow does one set a StrOpt that has a default to None (the value, not the string)? is that possible?20:02
jrollI feel like it is, but cannot find how20:02
efriedjroll: Is the default not None?20:08
jrollefried: correct20:10
jrollreading the code, it appears I'm wrong :(20:10
*** bobh has joined #openstack-oslo20:11
efriedjroll: Meaning you got it figured out?20:12
jrollefried: meaning I believe that I cannot set the value to None using the config file, and would love to be wrong about that :)20:12
efriedjroll: oic20:13
efriedjroll: So you want a config opt that's either not None by default, and to set that guy to None via the config (why?), or you care about this because you want a mutable opt to be able to go from some value back to its default of None.20:13
efriedjroll: In the latter case, does removing it from the conf do the trick?20:14
jrollefried: the former. see the help here: https://github.com/openstack/ironic/blob/master/ironic/conf/pxe.py#L3520:14
efriedhah20:14
efriedjroll: What about using literally '<None>'20:15
jrollefried: does not work20:15
efriedjroll: And you didn't even get lucky with the code doing an implicit falsey check. It's actually asking for `is None` :(20:18
efriedjroll: I assume you've tried20:19
efriedinstance_master_path =20:19
efriedand ended up with ''20:19
jrollefried: indeed20:19
efriedjroll: Well, this has only been a problem since 2014. What's the big deal?20:20
jrollefried: :D20:20
efriedjroll: Okay, a second pair of eyes has gone through both the ironic and oslo.config sides of this and confirmed yer scrood. Sorry.20:28
efriedjroll: The simplest fix would seem to be changing the ironic side to check for '' as well as None, since that seems like it surely covers the intent. The empty string is not a sensible value for that option, right?20:31
*** mmethot_ is now known as mmethot20:38
*** munimeha1 has quit IRC20:46
*** raildo has quit IRC21:06
jrollefried: thanks for the extra eyes. agree that seems reasonable, I'll see what other ironic folks say :)21:08
*** bobh has quit IRC21:20
*** pcaruana has quit IRC21:31
*** ansmith has quit IRC21:42
dhellmannjroll : why do you want to set a string option to None?21:45
*** bobh has joined #openstack-oslo21:46
openstackgerritEric Fried proposed openstack/oslo.service master: Limit monotonic to py2  https://review.openstack.org/61601321:47
efriedmordred: ^21:47
efrieddhellmann: https://github.com/openstack/ironic/blob/master/ironic/conf/pxe.py#L35 <==21:48
efrieddhellmann: (to answer the question about string option to None)21:48
efrieddhellmann: And we did check the rest of the ironic side code: it is actually doing 'is None' at the moment. Also the oslo code, which doesn't seem to provide a facility for explicitly setting to None.21:48
dhellmannit seems like None was a way to say "this is not set" and we can do that with the location information now21:49
openstackgerritEric Fried proposed openstack/oslo.service master: Fixture to mock loopingcall wait()  https://review.openstack.org/61597821:49
dhellmannyeah, None isn't a valid string so I don't think I'd want to add some complex logic to let the user spell out None in some format in the config file21:49
dhellmannit just feels like doing extra work, right?21:49
dhellmannhttps://docs.openstack.org/oslo.config/latest/reference/locations.html feels like a better way if the question is "did the user set this value?"21:50
efrieddhellmann: The consuming code seems crazy to me. I think any time you have a default that's a real string, it's reasonable not to make None a significant value.21:51
dhellmannI agree that it would be better to treat '' and None the same here21:51
dhellmannI think we weren't able to change the default of StringOpt because we have too many places looking for a None21:51
efried++, I think that's what jroll is gonna jroll with.21:51
efrieddhellmann: The default for StrOpt is indeed None.21:52
efriedwhich is fine.21:52
efriedIt's just that this case sets a real default.21:52
jrolldhellmann: I did not know about the location information. that may work, I'll need to dig into it. thanks!21:52
jrollwell, maybe not. but it's useful for other things :)21:53
dhellmannjroll : it's relatively new21:54
dhellmannlast cycle I think?21:54
jrollneat. I like it.21:54
*** geekinutah has quit IRC21:57
*** geekinutah has joined #openstack-oslo21:57
kmallocdhellmann: oslo.config simply ignores extra misc. data/options in config files right?21:58
kmalloccc anyone ^21:58
dhellmannkmalloc : yes21:58
* kmalloc is unclear on following deprecation policy, specifically removal of options from, say, keystonemiddleware if all the functionality is removed.21:58
kmalloce.g. pki/pkiz code is long dead, keystone hasn't supported the token formats in... a long time21:59
dhellmannI think the validator will tell you if you have data that isn't attached to a registered option, but the production code ignores things in the file until the app tries to access the data21:59
kmallocright. so, if we're removing all code that references the options... does it even make sense to keep them?21:59
dhellmannhave the options been marked as deprecated?21:59
kmallocno.21:59
kmallocthat is why i am asking21:59
dhellmannok21:59
kmallocthe code is removed22:00
kmallocand there is nothing that references it anywhere22:00
kmalloczero impact on running service22:00
dhellmannthe usual process is to mark the options as deprecated and then have the app emit warnings if the options are being set22:00
kmallocbleh22:00
dhellmannbut if the code is already gone and nothing uses them at all, you may not need to do that22:00
kmallocright22:00
kmallocwell it isn't gone yet22:00
kmallocbut it is bitrotting and untested in any real way22:00
dhellmannhmm22:00
efriedIf you wanted to be really nice, you could say "these were removed in XXX release and are ignored" and then delete them in the next release.22:01
kmallocthe code could not be used at all for ... like 2 yrs22:01
dhellmannso it's there but doesn't work?22:01
kmallocright22:01
kmallocit can't work with modern keystones22:01
dhellmannis it the sort of thing someone might have worked around without upstreaming that fix? which isn't to say i care that much about those people, but just for completeness...22:01
kmallocsince keystone does not emit pki/pkiz tokens22:01
kmallocunlikely.22:02
*** bobh has quit IRC22:02
dhellmannI think it's probably safe to just delete the stuff then22:02
kmallocthat was my view.22:02
kmallocbut i wanted to check and make sure i'm making the correct case for this22:02
dhellmannthe advance warning would have been the exception thrown from the broken code22:02
kmalloc++22:02
kmallocwell keystonemiddleware wouldn't receive a pki token anymore22:03
dhellmannit may still end up triggering a major version bump in the next release of the lib, but that's a separate conversation22:03
kmallockeystonemiddleware always gets a major bump, iirc or it used to22:03
dhellmannI'm not sure why it would always, but maybe it does22:03
kmallocbecause it is tied to library versions and can't be mixed and matched. has to be run in the same processspace as <service>22:03
kmallocit used to because it was versionlocked.22:04
kmalloce.g. it needs version X of keystoneclient (for example) and therefore nova would also only ever be able to use that version of ksc22:04
kmallocbecause it runs in the same processpace as nova.22:04
kmallocit may no longer have that.22:04
kmallocbut at one point it was managed like that, same as a service.22:04
kmallocdhellmann: *shrug*.22:06
dhellmannah, ok22:07
kmalloci think i can knock down the imports to KSA and a couple other minor things that should make it much less... obnoxious soon22:09
*** kgiusti has left #openstack-oslo22:09
kmallocideally, it would have oslo.config, keystoneauth and ... maybe webob?22:09
kmalloci'll figure that out once a lot of the dead code is removed :)22:09
*** dave-mccowan has quit IRC22:35
*** bobh has joined #openstack-oslo22:42
*** bobh has quit IRC22:45
*** bobh has joined #openstack-oslo22:46
*** bobh has quit IRC22:50
openstackgerritEric Fried proposed openstack/oslo.service master: Limit monotonic to py2  https://review.openstack.org/61601323:42
openstackgerritEric Fried proposed openstack/oslo.service master: Fixture to mock loopingcall wait()  https://review.openstack.org/61597823:43
efriedbnemec: Done, let's see if that flies...23:43

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!