Tuesday, 2014-07-01

openstackgerritJoshua Harlow proposed a change to openstack/taskflow: WBE response message validation  https://review.openstack.org/10369000:01
*** czzl has quit IRC00:22
*** markmcclain has quit IRC00:24
*** markmcclain has joined #openstack-oslo00:26
*** markmcclain has quit IRC00:27
*** markmcclain has joined #openstack-oslo00:27
*** praneshp has quit IRC00:27
*** erecio has joined #openstack-oslo00:28
zzzeekheya dhellmann : if we move forward with deeper SQLA extensions in oslo.db, what do you think of this change to tox.ini, is this feasible and how would downstream test runners need to change: http://paste.openstack.org/show/85190/00:30
zzzeeki tried other ways of making that work but right now you can’t have the conflicting requiremnents all as “deps” in tox00:31
openstackgerritBoris Pavlovic proposed a change to openstack/oslo.messaging: Enable cross service profiling  https://review.openstack.org/10327100:40
*** GheRivero is now known as GheRivero_afk00:40
openstackgerritBoris Pavlovic proposed a change to openstack/oslo.messaging: Enable cross service profiling  https://review.openstack.org/10327100:45
openstackgerritBoris Pavlovic proposed a change to openstack/oslo.messaging: Enable cross service profiling  https://review.openstack.org/10327100:49
*** markmcclain has quit IRC00:56
zzzeekwell im going to stick it in a gerrit tomorrow :)00:56
*** HenryG has quit IRC00:57
*** HenryG has joined #openstack-oslo00:59
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: WBE response message validation  https://review.openstack.org/10369001:04
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: WBE request message validation  https://review.openstack.org/10369501:04
*** dstanek is now known as dstanek_zzz01:06
*** jecarey has joined #openstack-oslo01:11
*** yamahata has joined #openstack-oslo01:18
*** erecio has quit IRC01:23
*** HenryG has quit IRC01:30
*** HenryG has joined #openstack-oslo01:30
openstackgerritxu-haiwei proposed a change to openstack/oslo-incubator: Fix .conf.sample file generation bug  https://review.openstack.org/10347301:32
*** dstanek_zzz is now known as dstanek01:36
openstackgerritlawrancejing proposed a change to openstack/oslo-incubator: Use immutable arg rather mutable arg  https://review.openstack.org/10369801:43
openstackgerritXavier Barbosa proposed a change to openstack/cliff: Inject default values to subcommands.  https://review.openstack.org/10370001:52
*** gcb has joined #openstack-oslo01:59
*** tongli has quit IRC02:24
openstackgerritA change was merged to openstack/oslo-incubator: Fix broken formatting of processutils.execute log statement  https://review.openstack.org/10356902:25
openstackgerritZhongyue Luo proposed a change to openstack/oslo-incubator: Use timestamp in loopingcall  https://review.openstack.org/10370402:26
*** zzzeek has quit IRC02:29
*** zhiyan_ is now known as zhiyan02:38
openstackgerritZhongyue Luo proposed a change to openstack/oslo-incubator: Use timestamp in loopingcall  https://review.openstack.org/10370402:42
*** dims_ has quit IRC02:46
*** zhiyan is now known as zhiyan_02:50
*** zhiyan_ is now known as zhiyan02:51
*** mriedem has left #openstack-oslo03:00
*** harlowja is now known as harlowja_away03:02
*** arnaud has quit IRC03:05
*** arnaud__ has quit IRC03:05
openstackgerritZhongyue Luo proposed a change to openstack/oslo-incubator: Use timestamp in loopingcall  https://review.openstack.org/10370403:07
*** harlowja_away is now known as harlowja03:09
*** praneshp has joined #openstack-oslo03:10
*** harlowja is now known as harlowja_away03:10
*** harlowja_away is now known as harlowja03:11
*** praneshp_ has joined #openstack-oslo03:13
*** praneshp has quit IRC03:16
*** praneshp_ is now known as praneshp03:16
*** HenryG has quit IRC03:17
*** HenryG has joined #openstack-oslo03:20
openstackgerritZhongyue Luo proposed a change to openstack/oslo-incubator: Changes calcuation of variable delay  https://review.openstack.org/10370803:27
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: Have the dispatch_job function return a future  https://review.openstack.org/10371203:34
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: WBE request message validation  https://review.openstack.org/10369503:45
*** gcb has quit IRC03:59
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: Add a conductor considerations section  https://review.openstack.org/10326704:05
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: Add a conductor considerations section  https://review.openstack.org/10326704:05
*** sileht has quit IRC04:06
*** rpodolyaka has joined #openstack-oslo04:06
*** sileht has joined #openstack-oslo04:09
*** jogo_awa1 has quit IRC04:11
*** ildikov has quit IRC04:11
*** pcm__ has quit IRC04:11
*** ekarlso has quit IRC04:11
*** SlickNik has quit IRC04:11
*** jd__ has quit IRC04:11
*** mrda has quit IRC04:12
*** arnaud has joined #openstack-oslo04:13
*** arnaud__ has joined #openstack-oslo04:14
*** erecio has joined #openstack-oslo04:21
*** mrda has joined #openstack-oslo04:22
*** jogo_awa1 has joined #openstack-oslo04:22
*** ildikov has joined #openstack-oslo04:22
*** pcm__ has joined #openstack-oslo04:22
*** ekarlso has joined #openstack-oslo04:22
*** SlickNik has joined #openstack-oslo04:22
*** jd__ has joined #openstack-oslo04:22
*** ildikov has quit IRC04:31
*** ildikov has joined #openstack-oslo04:32
*** jogo_awa1 has quit IRC04:32
*** jogo_away has joined #openstack-oslo04:32
*** ildikov has quit IRC04:49
*** erecio has quit IRC04:53
*** ajc_ has joined #openstack-oslo04:59
*** dstanek is now known as dstanek_zzz05:05
*** harlowja is now known as harlowja_away05:06
openstackgerritjichenjc proposed a change to openstack/oslo-incubator: Add lock_path as param to remove_external function  https://review.openstack.org/10373205:06
*** zzzeek has joined #openstack-oslo05:11
*** dstanek_zzz is now known as dstanek05:13
*** ildikov has joined #openstack-oslo05:23
*** dstanek is now known as dstanek_zzz05:28
*** tkelsey has joined #openstack-oslo05:32
openstackgerritPranesh Pandurangan proposed a change to openstack/taskflow: Convert usage of some state strings to objects  https://review.openstack.org/10338205:39
*** markmc has joined #openstack-oslo05:52
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: Add a conductor considerations section  https://review.openstack.org/10326706:06
*** mkoderer has joined #openstack-oslo06:24
*** dstanek_zzz is now known as dstanek06:29
*** SridharG has joined #openstack-oslo06:30
*** rpodolyaka has quit IRC06:43
*** zzzeek has quit IRC06:43
*** zzzeek has joined #openstack-oslo06:47
*** pblaho has joined #openstack-oslo06:52
*** zzzeek has quit IRC06:53
*** i159 has joined #openstack-oslo06:56
*** i159_ has joined #openstack-oslo07:02
*** i159 has quit IRC07:02
*** arnaud__ has quit IRC07:06
*** arnaud has quit IRC07:06
*** morganfainberg is now known as morganfainberg_Z07:12
*** arnaud has joined #openstack-oslo07:14
*** arnaud__ has joined #openstack-oslo07:15
*** arnaud__ has quit IRC07:20
*** arnaud has quit IRC07:20
*** arnaud has joined #openstack-oslo07:20
*** arnaud__ has joined #openstack-oslo07:20
*** dstanek is now known as dstanek_zzz07:24
*** praneshp has quit IRC07:27
*** rdopieralski has joined #openstack-oslo07:37
*** nacim has joined #openstack-oslo07:38
*** rdopieralski has quit IRC07:40
*** mrda is now known as mrda-away07:49
*** erecio has joined #openstack-oslo07:51
openstackgerritlawrancejing proposed a change to openstack/taskflow: Fix the section name in CONTRIBUTING.rst  https://review.openstack.org/10378007:51
*** flaper87|afk is now known as flaper8707:55
*** morganfainberg_Z is now known as morganfainberg07:55
*** ihrachyshka has joined #openstack-oslo08:08
boris-42markmc hi08:11
boris-42markmc I should write spec to https://github.com/openstack/oslo-specs ??08:11
boris-42markmc or oslo.messaging has separated repo?08:11
*** dstanek_zzz is now known as dstanek08:15
*** ihrachyshka has quit IRC08:15
*** ihrachyshka has joined #openstack-oslo08:16
*** nacim has quit IRC08:20
rpodolyaka1boris-42: oslo-specs repo is used to track specs to all projects within oslo program08:22
boris-42rpodolyaka1 ah great thanks08:23
rpodolyaka1boris-42: np08:23
*** erecio has quit IRC08:24
*** dstanek is now known as dstanek_zzz08:25
*** arnaud__ has quit IRC08:29
*** arnaud has quit IRC08:29
*** ujjain has quit IRC08:39
*** ujjain has joined #openstack-oslo08:39
*** ildikov has quit IRC08:42
openstackgerritgaryk proposed a change to openstack/oslo.vmware: Translations: make use of _LE, _LI and _LW  https://review.openstack.org/10182108:42
*** viktors has joined #openstack-oslo08:49
*** yamahata has quit IRC09:12
*** flaper87 is now known as flaper87|afk09:13
*** ihrachyshka has quit IRC09:13
openstackgerritA change was merged to openstack/taskflow: Clarify locked decorator is for instance methods  https://review.openstack.org/9869109:16
*** dstanek_zzz is now known as dstanek09:16
openstackgerritA change was merged to openstack/taskflow: Fix traces left in zookeeper  https://review.openstack.org/10041509:19
*** lbragstad_ has quit IRC09:22
*** lbragstad_ has joined #openstack-oslo09:24
*** dstanek is now known as dstanek_zzz09:26
*** oomichi has quit IRC09:30
*** dstanek_zzz is now known as dstanek09:48
openstackgerritBoris Pavlovic proposed a change to openstack/oslo-specs: Enable cross service profiling  https://review.openstack.org/10382509:51
boris-42markmc ^ I added spec09:52
openstackgerritA change was merged to openstack/oslo.messaging: Remove duplicate docs for MessageHandlingServer  https://review.openstack.org/10273109:57
*** dstanek is now known as dstanek_zzz09:58
*** ajo|mac has joined #openstack-oslo10:08
*** ihrachyshka has joined #openstack-oslo10:16
openstackgerritIlya Pekelny proposed a change to openstack/oslo.db: Opportunistic migration tests  https://review.openstack.org/9342410:18
openstackgerritIlya Pekelny proposed a change to openstack/oslo.db: Implementation Alembic as migration engine  https://review.openstack.org/9996510:18
*** dstanek_zzz is now known as dstanek10:49
*** openstack has joined #openstack-oslo10:51
openstackgerritBoris Pavlovic proposed a change to openstack/oslo.messaging: Enable cross service profiling  https://review.openstack.org/10327110:56
openstackgerritBoris Pavlovic proposed a change to openstack/oslo.messaging: Enable cross service profiling  https://review.openstack.org/10327110:57
*** dstanek is now known as dstanek_zzz10:59
*** jaosorior has joined #openstack-oslo11:06
jaosoriorAnybody here working with oslo.messaging?11:06
*** dims_ has joined #openstack-oslo11:07
ihrachyshkajaosorior: the question is ambiguous, but yes11:15
*** tkelsey has quit IRC11:17
*** erecio has joined #openstack-oslo11:22
*** nacim has joined #openstack-oslo11:33
dhellmannviktors: it looks like https://blueprints.launchpad.net/oslo/+spec/oslo-db-lib is pretty close to done (some of the TODO items like ACLs are done). Can you review it and update its status, please?11:41
viktorsdhellmann: hi! I think, that the single TODO is left - Update consumers11:45
viktorsnot all projects use  oslo.db - only keystone and neutron at the moment11:45
viktorscan I close this BP without the lats item&11:46
viktors?11:46
dhellmannviktors: yes, you've released the library so let's close that bp and use something else to track adoption (bps in each project, perhaps)11:49
dhellmannviktors: I'm going to remove that todo item from the template list on the wiki page11:50
*** dstanek_zzz is now known as dstanek11:50
viktorsdhellmann: ok, I'll close this BP11:50
dhellmannviktors: if you keep https://etherpad.openstack.org/p/juno-oslo-adoption-status up to date, I'll be happy with the tracking and the PTLs of the other projects can decide if they want blueprints or not11:51
viktorsdhellmann: oh, I missed this etherpad. Will do11:52
dhellmannviktors: when we move to storyboard, it will be a little easier to track all of this in one place, but I'm not sure about our ability to have cross-project bp dependencies in lp11:52
viktorsdhellmann: also we are suppose to release oslo.db 0.3.0 after we will get merged i159_'s patches with opportunistic migration tests for sqla-migrate and alembic11:56
dhellmannviktors: ok, thanks, I'll mention that today in the project meeting to give the other teams a little notice11:56
dhellmannviktors: do you have an estimate for when that will be? this week? next week?11:57
boris-42dhellmann Hi11:59
*** dstanek is now known as dstanek_zzz12:00
viktorsdhellmann: as soon as we'll merge some patches. I really hope, this week, but I'm not sure  :)12:00
viktorsAlso we with rpodolyaka1 also want to ask some question to Mike about  migration tests for alembic implementation.12:00
dhellmannviktors: ok, I'll say "soon" :-)12:00
viktorsdhellmann: :)12:00
*** rpodolyaka1 has left #openstack-oslo12:01
*** rpodolyaka1 has joined #openstack-oslo12:01
*** rpodolyaka1 is now known as rpodolyaka12:01
boris-42dhellmann hi=)12:01
*** rpodolyaka1 has joined #openstack-oslo12:03
*** rpodolyaka has quit IRC12:05
*** ajo|mac has joined #openstack-oslo12:06
*** rpodolyaka1 is now known as rpodolyaka12:06
*** viktors is now known as viktors|afk12:08
boris-42dhellmann could you pls take a look at my spec for osprofiler12:12
boris-42dhellmann https://review.openstack.org/#/c/103825/12:12
openstackgerritA change was merged to openstack/oslo-specs: Update the sphinx version of the module  https://review.openstack.org/10338712:13
*** ihrachyshka has quit IRC12:16
dhellmannboris-42: I'm about to head in to the office for the day, but I will look at it.12:16
boris-42dhellmann thanks12:16
jaosorioryeah, so is the coverage in oslo.messaging working for anyone? haven't gotten it to give results :/12:16
*** ajc_ has quit IRC12:17
boris-42markmc pls could you take a look at my spec as well..12:17
*** dhellmann is now known as dhellmann_12:19
*** dims_ has quit IRC12:32
*** dims_ has joined #openstack-oslo12:32
*** ihrachyshka has joined #openstack-oslo12:36
openstackgerritJuan Antonio Osorio Robles proposed a change to openstack/oslo.messaging: Raise if invalid conf is given to get_transport  https://review.openstack.org/10386612:41
*** tkelsey has joined #openstack-oslo12:43
openstackgerritChristian Berendt proposed a change to openstack-dev/cookiecutter: Use spaces instead of tabs for indentation  https://review.openstack.org/10386912:46
*** alexpilotti has joined #openstack-oslo12:47
*** dstanek_zzz is now known as dstanek12:50
*** SridharG has quit IRC12:53
openstackgerritChristian Berendt proposed a change to openstack-dev/oslo-cookiecutter: Use spaces instead of tabs for indentation  https://review.openstack.org/10387212:56
*** dstanek is now known as dstanek_zzz13:00
openstackgerritGordon Sim proposed a change to openstack/oslo-specs: Added spec for proposed AMQP 1.0 driver  https://review.openstack.org/9672913:02
*** pblaho_ has joined #openstack-oslo13:09
*** pblaho has quit IRC13:09
openstackgerritOpenStack Proposal Bot proposed a change to openstack/oslo.db: Updated from global requirements  https://review.openstack.org/10213213:11
*** markmcclain has joined #openstack-oslo13:15
*** mriedem has joined #openstack-oslo13:22
*** mriedem has left #openstack-oslo13:22
*** mriedem has quit IRC13:22
*** mriedem has joined #openstack-oslo13:22
openstackgerritYuriy Taraday proposed a change to openstack/oslo.config: Allow to reference values outside DEFAULT in substitution  https://review.openstack.org/10388413:23
*** jecarey has quit IRC13:31
*** pblaho_ is now known as pblaho13:33
*** pblaho_ has joined #openstack-oslo13:33
*** pblaho has quit IRC13:37
*** dstanek_zzz is now known as dstanek13:51
*** SridharG has joined #openstack-oslo13:58
*** pblaho_ is now known as pblaho14:02
*** zzzeek has joined #openstack-oslo14:04
*** flaper87|afk is now known as flaper8714:06
*** erecio has quit IRC14:19
*** nacim has quit IRC14:24
*** dims_ has quit IRC14:28
*** nacim has joined #openstack-oslo14:35
openstackgerritIlya Pekelny proposed a change to openstack/oslo.db: Add a base test case for DB schema comparison  https://review.openstack.org/9339814:39
*** dims_ has joined #openstack-oslo14:41
*** jecarey has joined #openstack-oslo14:52
*** james_li has joined #openstack-oslo14:55
*** flaper87 is now known as flaper87|afk14:57
*** Alexei_987 has joined #openstack-oslo14:58
rpodolyakazzzeek: hey! are you around?14:58
zzzeekheya14:59
rpodolyakaa quick question on alembic14:59
rpodolyakato make sure I understand things correctly14:59
rpodolyakasome time ago we had a patch to oslo.db to add an API layer to unify alembic/migrate migrations14:59
rpodolyakaso that I could pass a migrations repo dir and a Connection/Engine instance and run migrations scripts for this particular connection15:00
rpodolyakafor migrate this is quite straightforward15:00
rpodolyakabut for alembic we had to use some private API functions15:00
zzzeekthe connecvtion/engine setup is done by env.py15:01
rpodolyakato run migration scripts programatically15:01
zzzeekyou have control over that15:01
rpodolyakayep15:01
rpodolyakahttps://review.openstack.org/#/c/7187415:01
rpodolyakabut env.py is bound to a particular migration repo, right?15:01
rpodolyakaso the question is more like, I can't provide one within oslo.db15:01
*** dstanek is now known as dstanek_zzz15:02
rpodolyakaas I can't know in advance how it will be used in a consuming project like e.g. nova15:02
zzzeekthe front end env.py is in a migration repo but you can certainly control what it does externally15:02
zzzeeki dont see why you had to take apart command.upgrade here15:03
*** flaper87|afk is now known as flaper8715:03
rpodolyakaafair, we coundn't do something like: with environment.EnvironmentContext(...): upgrade()15:04
rpodolyakapassing an existing Connection instance15:04
zzzeekyou can put a Connection into the config and have your env.py read it15:05
zzzeekits so ironic that you say migrate is easier. the whole point of env.py was because of hte massive monkeypatching i had to do to get migrate to do the simplest things15:05
zzzeekwhen i used migrate, it called create_engine() in no less than three places15:05
rpodolyakaheh15:05
zzzeekit was utterly impossible to control where it did anything with connections and transactions were a total non starter15:06
rpodolyakayep15:06
zzzeekwith env.py i said, never again.  i will give a script to the user, that they can do anything with15:06
zzzeekso you can pass anything at all to env.py, you are the one calling it!15:06
zzzeekand writing it!15:06
*** viktors|afk is now known as viktors15:06
rpodolyakaok, so if I understand you correctly, you say that a consuming project must be somehow 'ready' for passing a connection to env.py?15:07
zzzeekit woudl be the most straightforward way, i could look some more and see other ways also15:07
zzzeekthe env.py is responsible for figuring out where the connection comes from15:08
zzzeekby deafult its looking in alembic.ini15:08
zzzeekim sure openstack projects dont use that15:08
*** dstanek_zzz is now known as dstanek15:09
rpodolyakayeah, I'm looking at neutron code right now and they use their own configs, parse them with oslo.config and create an Engine instance based on that url15:09
zzzeekim not sure how this patch woudl even work, woudlnt a non-cooperating env.py just override the connection you’re putting in there?15:09
zzzeekprojects using oslo.db should absolutely be using an oslo.db env.py, their own env.py should be:   “from oslo.db import env;  env.run()”15:10
zzzeekjust like you see in everyone’s setup.py using “pbr”15:10
rpodolyakacool, we should add one :)15:10
zzzeekthat’s waht i usually do, the env.py in the migrate dir just becomes a stub15:10
zzzeekpulls in a more authoriative env.py from elsewhere15:10
rpodolyakaseems that we'll need this for https://review.openstack.org/#/c/99965/13/oslo/db/sqlalchemy/test_migrations.py15:14
rpodolyakaand until consuming projects don't cooperate with us (until we don't provide common env.py), it seems that we'll need to monkey-patch projects env.py in tests15:14
rpodolyakain order to pass the Engine instance we want15:15
zzzeekOK well env.py is totally responsible for setting up environment.  here we’re getting into, you’re trying to skip that.  and there’s no API for skipping env.py15:15
zzzeekb.c. the idea of env.py is that there could be totally crazy things going on like, it connects to three different DBs and runs them all.   but.   what if a program wants to override env.py, lets look15:15
zzzeekall you’d really want is to be able to inject ScriptDirectory.run_env()15:17
zzzeeknot a big deal15:17
*** praneshp has joined #openstack-oslo15:20
*** i159_ has quit IRC15:21
*** ihrachyshka has quit IRC15:24
rpodolyakazzzeek: ok, will take a look alembic API once again, thanks a lot!15:31
zzzeekrpodolyaka: if we need enhancement to change run_env(), let me know15:32
rpodolyakazzzeek: ok15:32
zzzeekrpodolyaka: i think having everyone use oslo.db.env.py is best but we can do either15:32
*** erecio has joined #openstack-oslo15:33
*** eezhova has joined #openstack-oslo15:36
*** flaper87 is now known as flaper87|afk15:40
*** flaper87|afk is now known as flaper8715:42
*** dstanek is now known as dstanek_zzz15:47
*** nacim has quit IRC15:49
*** markmcclain has quit IRC15:52
*** markmc has quit IRC15:55
*** praneshp has quit IRC15:57
*** praneshp has joined #openstack-oslo15:59
*** stannie1 has joined #openstack-oslo16:01
*** stannie1 is now known as stannie4216:01
*** viktors is now known as viktors|afk16:06
openstackgerritGraham Hayes proposed a change to openstack/oslosphinx: Added a incubating project config option  https://review.openstack.org/10393516:10
*** zhiyan is now known as zhiyan_16:19
*** arnaud has joined #openstack-oslo16:20
*** erecio has quit IRC16:21
*** dstanek_zzz is now known as dstanek16:28
*** arnaud has quit IRC16:30
*** arnaud has joined #openstack-oslo16:33
*** Alexei_987 has quit IRC16:37
*** tkelsey has quit IRC16:40
*** dims_ has quit IRC16:41
*** dims_ has joined #openstack-oslo16:42
*** erecio has joined #openstack-oslo16:48
*** zzzeek has quit IRC16:50
*** jaosorior has quit IRC16:52
*** pblaho has quit IRC16:56
*** ihrachyshka has joined #openstack-oslo16:56
*** arnaud has quit IRC16:57
openstackgerritPranesh Pandurangan proposed a change to openstack/taskflow: Convert usage of some state strings to objects  https://review.openstack.org/10338216:59
*** markmcclain has joined #openstack-oslo17:01
*** markmcclain1 has joined #openstack-oslo17:03
*** markmcclain has quit IRC17:05
*** harlowja_away is now known as harlowja17:06
*** flaper87 is now known as flaper87|afk17:06
openstackgerritPranesh Pandurangan proposed a change to openstack/taskflow: Convert usage of some state strings to objects  https://review.openstack.org/10338217:07
*** rpodolyaka1 has joined #openstack-oslo17:18
*** james_li has quit IRC17:21
openstackgerritPranesh Pandurangan proposed a change to openstack/taskflow: Convert usage of some state strings to objects  https://review.openstack.org/10338217:22
*** dstanek is now known as dstanek_zzz17:22
boris-42dhellmann_ hi17:23
*** rpodolyaka1 has quit IRC17:23
*** dstanek_zzz is now known as dstanek17:25
*** lbragstad_ is now known as lbragstad17:31
*** tmcpeak has joined #openstack-oslo17:32
tmcpeakI know jogo is the main guy to talk to for hacking stuff, but I've thrown together a PoC for a security check, was wondering if anybody can tell me if I'm headed in the right direction and help me sort out some next steps17:35
*** erecio has quit IRC17:35
*** pblaho has joined #openstack-oslo17:39
*** arnaud has joined #openstack-oslo17:48
*** arnaud__ has joined #openstack-oslo17:48
*** rpodolyaka1 has joined #openstack-oslo17:49
*** erecio has joined #openstack-oslo17:50
*** dhellmann_ is now known as dhellmann17:53
*** markmcclain1 has quit IRC17:53
*** markmcclain has joined #openstack-oslo17:53
*** tongli has joined #openstack-oslo17:56
tonglitrying to use oslo.messaging , producer post a message, wanting to have 2 or more consumers to consume same messages.17:57
openstackgerritDavanum Srinivas (dims) proposed a change to openstack/oslo-incubator: Ability to customize default_log_levels for each project  https://review.openstack.org/9400117:57
tonglican anyone tell me how this is done using oslo.messaging?17:58
*** zzzeek has joined #openstack-oslo18:00
*** rpodolyaka1 has quit IRC18:00
*** harlowja is now known as harlowja_away18:10
tmcpeakjogo_away: you really away? :)18:12
*** arnaud__ has quit IRC18:13
*** arnaud has quit IRC18:13
*** tmcpeak has left #openstack-oslo18:21
openstackgerritSean Dague proposed a change to openstack/oslo-incubator: update new requests logger to default WARN  https://review.openstack.org/10396018:26
*** erecio has quit IRC18:28
*** james_li has joined #openstack-oslo18:29
dhellmanndims, flaper87|afk, markmc, jd__, or haypo : do any of you have time for a spot check of this spec? https://review.openstack.org/#/c/97296/418:30
boris-42dhellmann I know you are busy a lot18:31
openstackgerritA change was merged to openstack/oslo-specs: Add API details section to graduation spec  https://review.openstack.org/10226918:31
boris-42dhellmann but it will be soon feature freeze and all that crap18:31
boris-42dhellmann and this small patch in oslo.messaging is essential -> to be able to show others to osprofiler works18:32
dhellmannboris-42: sorry, I was out yesterday so I'm still catching up with everything I missed18:32
boris-42dhellmann yep sure =)18:32
*** harlowja_away is now known as harlowja18:33
*** erecio has joined #openstack-oslo18:37
*** ihrachyshka has quit IRC18:37
*** ajo|mac has joined #openstack-oslo18:41
boris-42dhellmann I really don't know why people is afraid so18:42
boris-42dhellmann about profiler18:42
boris-42dhellmann it doesn't add any overhead if you don't use it...18:43
boris-42dhellmann even if it is turned on..18:43
dhellmannboris-42: because not everyone has time to read the code to verify that claim18:43
*** arnaud has joined #openstack-oslo18:44
boris-42dhellmann I replied on your comments18:49
*** ajo|mac has quit IRC18:50
boris-42dhellmann should I fix somehow description?18:51
*** jecarey has quit IRC18:54
*** alexpilotti has quit IRC18:57
boris-42dhellmann so actually I should change API18:59
boris-42dhellmann in osprofiler18:59
boris-42dhellmann to move one patch from oslo.messaging18:59
dhellmannboris-42: please put this info in the spec or the review comments19:00
boris-42dhellmann but it's just 0.1.3 not 0.2.0 version19:00
boris-42dhellmann I already I am going to put in spec19:00
dhellmannboris-42: ok19:01
boris-42dhellmann is it point one clear now?19:01
boris-42dhellmann about message?19:01
dhellmannboris-42: I'll have to get back around to look at the spec again in a bit19:01
harlowjazzzeek where u located (out of curosity)19:06
zzzeekbrooklyn ny19:06
harlowjanice, good ole brooklyn19:06
zzzeekcome on over there’s great bars here19:08
openstackgerritBoris Pavlovic proposed a change to openstack/oslo-specs: Enable cross service profiling  https://review.openstack.org/10382519:08
morganfainbergzzzeek, wanted to ask you, do you prefer when people use sqlalchemy.and_ / .or_ + session.query().filter or using the session.query().filter_by mechanisms?19:12
*** ildikov has joined #openstack-oslo19:12
zzzeekdo you mean multple filter calls vs. using and_() ?19:12
*** jecarey has joined #openstack-oslo19:12
morganfainbergzzzeek, .filter_by(thing=<value>) or .filter(and_(MODEL.id == <val>, etc))19:13
zzzeekthey are all fine, filter_by() is a little less reliable, fixed some bugs in it recently19:13
morganfainbergright19:14
morganfainbergok19:14
morganfainbergi prefer filter, but flake8 balks at 'Model.Column == None'19:14
morganfainbergsays you need to use 'is None'19:14
zzzeekwe should remove that flake8 rule19:14
zzzeekive seen it removed in some suites already19:14
morganfainbergzzzeek, i'll propse that on the ML19:15
morganfainbergpropose*19:15
zzzeekif you really want to avoid the == you can say Column.is_(None)19:15
zzzeekor Column == sql.null()19:15
zzzeekor Column.is_(sql.null())19:15
morganfainbergah19:15
zzzeek:)19:15
zzzeekall of which are more of a PITA19:15
morganfainbergthat might be a good alternative19:15
zzzeekim not 100% sure those are all more backend-agnostic.  i think they are.  true()/ false() are more flaky19:15
zzzeeklike if you say column.is_(true())19:16
zzzeekthat might fail on MySQL19:16
morganfainbergzzzeek, have to say it's kinda cool being our own downstream :P19:16
zzzeekyeah19:16
morganfainbergget to ask all these questions :)19:16
*** ihrachyshka has joined #openstack-oslo19:16
morganfainbergi think the .and_() and .or_() + .filter() is way easier to read in code, which is why i was asking19:16
openstackgerritMichael Bayer proposed a change to openstack/oslo.db: Test for distinct SQLAlchemy major releases  https://review.openstack.org/10397519:16
zzzeekOK who wants to be a reviewer on that19:17
morganfainbergzzzeek, anyway thanks!19:17
morganfainbergzzzeek, i need lunch :P catch ya a bit later on19:17
zzzeeklater19:17
*** arnaud has quit IRC19:20
*** ildikov has quit IRC19:20
openstackgerritMichael Bayer proposed a change to openstack/oslo.db: Test for distinct SQLAlchemy major releases  https://review.openstack.org/10397519:21
ihrachyshkazzzeek, what,s your timezone? I have stuff to discuss with you.19:21
zzzeekEDT19:21
ihrachyshkaok, meaning you'll be avail the next hour :) correct?19:23
zzzeekIf im still sitting here…. :)19:23
zzzeekwhat are we talking about19:23
zzzeekand/or do you want to send me a review or somethign up front19:23
ihrachyshkamore of discussion regarding db lock timeouts we experience in neutron19:24
zzzeekok19:24
zzzeekmy short answer to that is, “dont use locks” :)19:25
ihrachyshkayou can check https://review.openstack.org/103540 to get the history of our failure :)19:28
harlowjazzzeek when i get back in NY might take u up on that (most of my family is in NYC or upstate, aka the rest of NY, ha)19:29
dhellmannflaper87|afk: the issue with duplicate option errors is likely caused by those options not having exactly the same settings (check the help text, for example)19:30
dhellmannflaper87|afk: let me know if syncing the incubator into glance doesn't fix it19:30
*** markmcclain has quit IRC19:34
zzzeekihrachyshka: my immediate thought there is that you are seeing traditional deadlocks.  it’s not specific to using eventlet, though eventlet’s timing may increase the chance of such a thing happening.19:34
zzzeekconcurrent transactions happen regardless of eventlet or even threads being in use19:34
zzzeekihrachyshka: although, I guess if the other side of the deadlock is that eventlet won’t context switch back until the second actor is done, then yes eventlet is a player here, OK19:35
ihrachyshkazzzeek, how so? if mysqldb blocks thread switch, where do concurrent transactions come from?19:35
zzzeeka deadlock can occur just with two mutually dependent locks, shared between two processes.19:36
zzzeekbut youre rgiht, here, its just one lock and the other “lock” is eventlet not context switching back19:36
zzzeekbut, hmm19:37
zzzeekwouldnt eventlet go to thread #2, do the thing, hit the lock, then that’s blocking IO, so it switches back again?19:37
ihrachyshkait locks deep in c code in mysqldb, so no event19:38
zzzeekOK19:38
ihrachyshkaet monkey patching19:38
zzzeekwell then this goes back to waht I’ve talked about a few weeks ago somehwere, one sec19:38
zzzeeklook into alternative drivers: http://www.diamondtin.com/2014/sqlalchemy-gevent-mysql-python-drivers-comparison/19:38
ihrachyshkaaha, that's what I actually searched for19:39
zzzeekor try to work aroudn the lock.  are you using SELECT FOR UPDATE ?19:39
ihrachyshkayes, we do use it19:39
zzzeekis that the source of the row lock ?19:39
ihrachyshkalet me show one of the patches to workaround a particular deadlock scenario19:40
ihrachyshkait's https://review.openstack.org/10093419:41
ihrachyshkayes, this is probably the source, we have multiple locks like that in neutron (don't ask me why) :)19:41
harlowjazzzeek if u interested, https://bitbucket.org/eventlet/eventlet/pull-request/29/fix-use-of-semaphore-with-tpool-issue-137/diff#comment-126671 is the known issue with eventlet (i believe this is the right link)19:42
zzzeekis the idea, send notifcations outside of transaction blocks ?19:42
harlowjachris (comstud) can be found in the #openstack-nova channel afaik19:42
ihrachyshkayes, it's the idea19:42
ihrachyshkabut the patch is ugly, so I don't want to extend the approach19:42
zzzeekihrachyshka: if you send a notifaction outside of the transaction block, then that whole thread is done, isnt the issue that the sending of that notifcation will interrupt some other thread?  which then could be contending with a third?19:43
zzzeekihrachyshka: im not deeply experienced with async so i could be misunderstanding.   but this code pattern, you could clean it up so that it isn’t “ugly”19:45
zzzeeke.g. build it into “context” and the context manager19:45
zzzeekjust like a web framework, add “post-transaction” callables19:45
ihrachyshkaI guess thread1 opens transaction, locks on a row, then triggers rpc, it yields to thread2 that also locks on the row, the latter is blocked, but mysqdb does not yield since it's not under eventlet control19:46
zzzeekso inside of self.disassociate_floatingips() -> context.post_transaction(self.notify_routers_updated, context, router_ids)19:46
ihrachyshkazzzeek, that's exactly what my spec I've sent before is about :)19:47
zzzeekihrachyshka: OK….. hmm so you know for sure that context does not siwtch between thread1-> transaction completed and “triggers rpc” because there are no blocking operations19:47
ihrachyshkahm. I didn't get the last one19:48
boris-42hey somebody19:48
boris-42stupid question19:48
boris-42when we are testing for example cinder19:49
boris-42we are using oslo.messaging from master in cinder gates?19:49
ihrachyshkaI know that in neutron, if you call anything that can yield under transaction, it may result in db lock timeout19:49
zzzeekihrachyshka: im trying to confirm in my head why we know that, thread1 completes transaction, thread2 starts new transaction, locks row, thread1 sends RPC, thread2 is bumped by the RPC call, thread3 then starts up, locks row, deadlock19:49
boris-42or not19:49
zzzeekand i think the reason is, thread2 would never start before thread1 is finished because thread1 does not block on IO at that point19:49
zzzeekihrachyshka: or, RPC only interrupts the thread that it is actually called from19:50
ihrachyshkathread1 hasn't completed transaction, it yielded to thread2 by calling to rpc19:50
zzzeekihrachyshka: yes, but im saying, with the fix in place19:50
zzzeekihrachyshka: where RPC call is outside of transaction19:50
ihrachyshkaif thread2 opens transaction, there is no way for it to yield to another thread, unless we call to smth that can yield. and we try to avoid that.19:52
zzzeekihrachyshka: but beyond that, if you already know that’s the fix, I think the pattern should be:   “with context.transaction() as session:   …. do stuff  …  context.post_exec(fn, *args)”19:52
zzzeekihrachyshka: yes that was what i was asking19:52
zzzeekihrachyshka: e.g. that when we deal with eventlet, it isn’t like traditional threads that are totlaly non-deterministic19:52
zzzeekihrachyshka: we know when it might be context switching19:52
zzzeekihrachyshka: which is odd!  even though this is “implicit” async, it’s still more explicit than regular threads19:53
ihrachyshkathe fix is proved to be correct in local testing. and adding that post_exec thing is my goal with the spec19:53
ihrachyshkayes, eventlet is explicit19:53
zzzeekihrachyshka: yes.  i think code like this should have zero SQLAlcehmy-specific transaction code and context managers should handle all that.19:53
dhellmannboris-42: the integration tests use master of oslo.messaging, but the unit tests do not19:53
zzzeekihrachyshka: “implicit” vs. “explicit” async is, gevent/eventlet vs. twisted / tornado / node.js19:53
ihrachyshkait's either you call eventlet.sleep() or you call a monkey patched functiom, no randomness19:54
zzzeekihrachyshka: gottit19:54
zzzeekihrachyshka: there’s also an after_commit() event on the Session19:54
zzzeekihrachyshka: if you wanted to leverage that19:54
zzzeekihrachyshka: but id make that happen behind the scenes19:54
zzzeekihrachyshka: also, this all should be an oslo.db object19:55
ihrachyshka+ for after_commit hook, sounds like what I may need19:55
zzzeekihrachyshka: OK but dont litter all the code with @event.listen() :)   do that in just one place19:56
ihrachyshkawhat should be oslo.db object?19:56
zzzeekihrachyshka: this may be longer term, but ideally id like projects to say: from oslo.db import transaction;   “with tranasction() as session:”19:56
ihrachyshkayou mean, implement it in oslo.db and then use in neutron? ok19:57
zzzeekany kind of database pattern that is invented by a project should be maintained in oslo.db and called from there19:57
zzzeekihrachyshka: yes19:57
zzzeekwe need to take as much SQLAlchemy “creational” code as possible out of projects19:58
zzzeekihrachyshka: projects will of coruse be using query(), and flush() if they have to, even execute(), but the scoping and framework of things should follow shared patterns19:58
*** arnaud has joined #openstack-oslo19:58
ihrachyshkaok, so good pointers and discussion so far :) I think I can continue from here. thanks a lot and welcome on openstack board :) we need your expertise like hell.19:59
boris-42dhellmann ok, for me its important that dsvm are using master20:00
zzzeekglad i could help20:00
boris-42dhellmann so we will be able to use osprofiler in gates even if there will be no new release of oslo.messaging for a while20:00
*** tcammann has joined #openstack-oslo20:00
dhellmannboris-42: now that we can make alpha releases, we can do that every week or so20:00
boris-42dhellmann ah ok20:01
boris-42dhellmann so I added extra details in spec that you asked*20:01
*** dims__ has joined #openstack-oslo20:15
tcammannHello looking for a core reviewer to +2/+A a change thats been sitting around a while with few +1s https://review.openstack.org/#/c/99695/20:16
*** markmcclain has joined #openstack-oslo20:16
*** dims_ has quit IRC20:18
*** dims__ has quit IRC20:20
*** erecio has quit IRC20:21
*** ajo|mac has joined #openstack-oslo20:29
*** dims__ has joined #openstack-oslo20:33
*** ihrachyshka has quit IRC20:40
*** dims__ has quit IRC20:41
*** ajo|mac has quit IRC20:45
*** ajo|mac has joined #openstack-oslo20:47
*** pblaho has quit IRC20:53
openstackgerritA change was merged to openstack/taskflow: Make the expiring cache a top level cache type  https://review.openstack.org/9856521:13
*** ajo|mac_ has joined #openstack-oslo21:13
*** ajo|mac has quit IRC21:15
*** SridharG has quit IRC21:33
*** dims__ has joined #openstack-oslo21:36
*** james_li has quit IRC21:37
*** markmcclain has quit IRC21:42
*** stannie42 has left #openstack-oslo21:49
*** mrda-away is now known as mrda22:02
*** zzzeek has quit IRC22:14
*** james_li has joined #openstack-oslo22:15
*** pcm__ has quit IRC22:15
*** dstanek is now known as dstanek_zzz22:28
*** james_li has quit IRC22:43
*** dims__ has quit IRC22:44
*** dstanek_zzz is now known as dstanek22:49
*** tcammann has quit IRC22:54
*** dims__ has joined #openstack-oslo22:58
*** oomichi has joined #openstack-oslo23:07
openstackgerritJoshua Harlow proposed a change to openstack/taskflow: Add basic WBE validation sanity tests  https://review.openstack.org/10401623:19
openstackgerritOpenStack Proposal Bot proposed a change to openstack/oslo-incubator: Updated from global requirements  https://review.openstack.org/10402223:36
openstackgerritJoshua Harlow proposed a change to openstack/oslo.messaging: Add a thread + futures executor based executor  https://review.openstack.org/7091423:42

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!