Friday, 2018-02-09

*** itlinux has joined #openstack-glance00:06
*** threestrands has joined #openstack-glance00:27
*** threestrands has quit IRC00:27
*** threestrands has joined #openstack-glance00:27
*** itlinux has quit IRC00:34
*** zhurong has joined #openstack-glance00:47
*** harlowja has joined #openstack-glance00:57
*** dalgaaf has quit IRC01:09
*** dalgaaf has joined #openstack-glance01:10
*** zhurong has quit IRC01:11
*** zhurong has joined #openstack-glance01:14
*** threestrands has quit IRC01:29
*** Nil_ has quit IRC01:38
*** r-daneel has quit IRC01:45
*** namnh has joined #openstack-glance02:37
*** ChanServ changes topic to "OpenStack Glance"03:10
*** ChanServ changes topic to "OpenStack Glance | priorities: http://lists.openstack.org/pipermail/openstack-dev/2018-February/127181.html"03:10
*** ChanServ changes topic to "OpenStack Glance | priorities: http://lists.openstack.org/pipermail/openstack-dev/2018-February/127181.html | logs at http://eavesdrop.openstack.org/irclogs/%23openstack-glance/"03:10
*** nicolasbock has quit IRC03:39
*** nicolasbock has joined #openstack-glance03:49
*** nicolasbock has quit IRC03:56
*** rosmaita has quit IRC04:02
*** udesale has joined #openstack-glance04:02
*** namnh has quit IRC04:13
*** abhishekk has joined #openstack-glance04:22
*** zhurong has quit IRC04:35
Shilpaabhishekk: GM04:42
abhishekkShilpa, hi, good morning04:47
Shilpacan we discuss LP1747869 now?04:47
abhishekkyes04:49
*** links has joined #openstack-glance04:52
abhishekkShilpa, is it ok if we discuss it after an hour?04:53
Shilpaok04:53
*** links has quit IRC04:54
*** links has joined #openstack-glance05:03
*** links has quit IRC05:10
*** harlowja has quit IRC05:10
*** links has joined #openstack-glance05:12
openstackgerritbaiwenteng proposed openstack/glance master: Fix typos in glance  https://review.openstack.org/54259705:23
*** pdeore has joined #openstack-glance05:30
*** harlowja has joined #openstack-glance05:35
*** zhurong has joined #openstack-glance05:36
*** harlowja has quit IRC05:39
*** ratailor has joined #openstack-glance05:40
*** pgadige__ has joined #openstack-glance05:49
*** itlinux has joined #openstack-glance05:58
*** e0ne has joined #openstack-glance06:05
*** e0ne has quit IRC06:06
openstackgerritXianjin Shao proposed openstack/glance master: Description about the parmeters 'op', 'path' and 'value' when update image property or metadata  https://review.openstack.org/54150806:26
*** itlinux has quit IRC06:27
*** MattMan has quit IRC06:30
*** pbourke has quit IRC06:30
openstackgerritOpenStack Proposal Bot proposed openstack/glance master: Imported Translations from Zanata  https://review.openstack.org/54264207:03
*** namnh has joined #openstack-glance07:03
*** rcernin has quit IRC07:17
*** AlexeyAbashkin has joined #openstack-glance07:20
*** e0ne has joined #openstack-glance07:27
openstackgerritXianjin Shao proposed openstack/glance master: Description about the parmeters 'op', 'path' and 'value' when update image property or metadata  https://review.openstack.org/54150807:28
*** pcaruana has joined #openstack-glance07:35
*** ykarel has joined #openstack-glance07:42
ykareljokke_, hi07:43
ykareljokke_, please check my comment in your merged patch: https://review.openstack.org/#/c/54174007:44
*** AlexeyAbashkin has quit IRC07:44
*** alexchadin has joined #openstack-glance07:48
*** AlexeyAbashkin has joined #openstack-glance07:54
Shilpaabhishekk: is it ok to discuss LP1747869 now?07:55
openstackgerrityatin proposed openstack/glance master: Fix wrong usage of extend in list_image_import_opts  https://review.openstack.org/54269907:58
*** MattMan has joined #openstack-glance08:01
openstackgerrityatin proposed openstack/glance master: Fix wrong usage of extend in list_image_import_opts  https://review.openstack.org/54269908:02
*** dtantsur|afk is now known as dtantsur08:08
*** mosulica has joined #openstack-glance08:09
openstackgerritXianjin Shao proposed openstack/glance master: Description about the parmeters 'op', 'path' and 'value' when update image property or metadata  https://review.openstack.org/54150808:25
*** tesseract has joined #openstack-glance08:26
*** namnh has quit IRC08:27
*** d0ugal has quit IRC08:33
*** d0ugal has joined #openstack-glance08:47
openstackgerritShilpa Devharakar proposed openstack/glance master: Add validation to check if E-M-C is already in sync  https://review.openstack.org/54073608:54
*** pbourke has joined #openstack-glance08:58
*** alexchadin has quit IRC09:07
*** alexchadin has joined #openstack-glance09:08
*** ykarel is now known as ykarel|lunch09:08
*** zhurong has quit IRC09:16
openstackgerritXianjin Shao proposed openstack/glance master: Description about the parmeters 'op', 'path' and 'value' when update image property or metadata  https://review.openstack.org/54150809:19
abhishekkjokke_, you around09:24
jokke_yeah09:24
abhishekkjokke_, please have a look at https://review.openstack.org/#/c/542699/2, its important we need to backport it as well09:24
openstackgerrityatin proposed openstack/glance master: Fix wrong usage of extend in list_image_import_opts  https://review.openstack.org/54269909:26
jokke_yep ... it seems like our config refresher did not have that patch it ...09:32
abhishekkright09:32
jokke_just waiting for the gate09:34
jokke_ykarel|lunch: thanks, nice catch09:35
*** edmondsw has joined #openstack-glance09:40
abhishekkjokke_, ok09:41
abhishekkjokke_, Pushing one patch for postgresql fix, I will set workflow to -1 initially so that you and rosmaita can decide whether its good approach or not09:42
jokke_abhishekk: thanks, I like it already09:44
abhishekk:D09:44
*** edmondsw has quit IRC09:45
abhishekkShilpa, hi I am sorry,I haven't saw your message09:49
Shilpaok, actually i am also working on solution to resovle postgresql fix, and before finalizing, need to discuss the same with you09:49
abhishekkShilpa, wait for sometime, I am proposing a patch soon to fix it09:50
abhishekktill then you can ask me your doubts09:50
Shilpais glance supports any other engine for offline upgrade other than ,ysql/sqlite/postgres09:52
Shilpamysql*09:52
Shilpaor we need to consider issue for postgres only?09:53
*** AlexeyAbashkin has quit IRC09:56
abhishekkno consider postgresql only09:56
*** AlexeyAbashkin has joined #openstack-glance09:57
Shilpaok09:59
Shilpaas of now i have verified configuring postgres and verifying running EMC scripts on it, and they are executing with no errors09:59
Shilpai have also verified original defect, upgrading from ocata to pike on postgres and yes that also success10:00
abhishekkShilpa, that  is because in EMC script we don't have any constraint which will not work on postgresql10:00
abhishekkIf you add that constraint then EMC will fail for postgresql10:00
openstackgerritAbhishek Kekane proposed openstack/glance master: Offline migration support for postgresql  https://review.openstack.org/54275610:06
*** bhagyashri_s has joined #openstack-glance10:06
*** nsingh has joined #openstack-glance10:06
*** pooja-jadhav has joined #openstack-glance10:06
*** pooja_jadhav has quit IRC10:07
*** Shilpa has quit IRC10:07
*** Shilpa has joined #openstack-glance10:07
*** bhagyashris has quit IRC10:08
*** niraj_singh has quit IRC10:08
abhishekkShilpa, ^^^10:08
Shilpasorry Abhishekk, got disconnected10:08
abhishekkjokke_, ^^^10:08
abhishekkShilpa, no issues, I have just pushed a patch, please have a look10:08
Shilpaok, sure10:09
*** ShilpaSD has joined #openstack-glance10:09
*** jcfro- has joined #openstack-glance10:11
jcfro-(_)                                               | |10:11
jcfro-(_)                                               | |10:11
jcfro-(_)                                               | |10:11
*** ykarel|lunch is now known as ykarel10:11
jcfro-_ _ __ ___   ___ _   _ _ __   ___ _ __ _ __   ___| |_ ___   ___  _ __ __ _10:11
jcfro-_ _ __ ___   ___ _   _ _ __   ___ _ __ _ __   ___| |_ ___   ___  _ __ __ _10:11
jcfro-_ _ __ ___   ___ _   _ _ __   ___ _ __ _ __   ___| |_ ___   ___  _ __ __ _10:11
jcfro-| | '__/ __| / __| | | | '_ \ / _ \ '__| '_ \ / _ \ __/ __| / _ \| '__/ _` |10:11
jcfro-| | '__/ __| / __| | | | '_ \ / _ \ '__| '_ \ / _ \ __/ __| / _ \| '__/ _` |10:11
jcfro-| | '__/ __| / __| | | | '_ \ / _ \ '__| '_ \ / _ \ __/ __| / _ \| '__/ _` |10:11
jcfro-| | | | (__ _\__ \ |_| | |_) |  __/ |  | | | |  __/ |_\__ \| (_) | | | (_| |10:11
jcfro-| | | | (__ _\__ \ |_| | |_) |  __/ |  | | | |  __/ |_\__ \| (_) | | | (_| |10:11
jcfro-| | | | (__ _\__ \ |_| | |_) |  __/ |  | | | |  __/ |_\__ \| (_) | | | (_| |10:11
jcfro-|_|_|  \___(_)___/\__,_| .__/ \___|_|  |_| |_|\___|\__|___(_)___/|_|  \__, |10:11
jcfro-|_|_|  \___(_)___/\__,_| .__/ \___|_|  |_| |_|\___|\__|___(_)___/|_|  \__, |10:11
jcfro-|_|_|  \___(_)___/\__,_| .__/ \___|_|  |_| |_|\___|\__|___(_)___/|_|  \__, |10:11
jcfro-| |                                             __/ |10:12
jcfro-| |                                             __/ |10:12
jcfro-| |                                             __/ |10:12
jcfro-|_|                                            |___/10:12
jcfro-|_|                                            |___/10:12
jcfro-|_|                                            |___/10:12
jcfro-ShilpaSD Shilpa pooja-jadhav nsingh bhagyashri_s AlexeyAbashkin alexchadin pbourke d0ugal tesseract mosulica MattMan ykarel|lunch pcaruana e0ne pgadige__ ratailor pdeore links abhishekk udesale dalgaaf mvk_ gaurangt lbragstad DuncanT cargonza openstackgerrit esberglu tbarron sapd mnaser _alastor__ hoonetorg masber pdr_ idlemind jdillaman pliu wxy NostawRm mtreinish jungleboyj jdurgin kairat zzzeek keekz twouters clayton amea10:12
*** jcfro- has quit IRC10:12
ShilpaSDabhishekk: Hi10:12
abhishekkShilpaSD, hi10:13
ShilpaSDsaw your changes10:13
ShilpaSDconstraint issue will be there for postgras, right10:14
abhishekkShilpaSD, yes in case of online migrations10:15
ShilpaSDbut online migrations always support MySQL, right10:15
abhishekkyes10:16
ShilpaSDbut during oflline migrations also issue of constraint will be there10:16
ShilpaSDfor postgreSQL10:17
*** pgadige__ has left #openstack-glance10:17
abhishekkShilpa, yes but during offline migration operator can resolve those issues and it will not break anything10:17
ShilpaSDSo operators take care of any constraints or other issues during oflline migrations10:18
abhishekkyes, without affection production environment which is not possible if it is online10:19
abhishekks/affection/affecting10:19
ShilpaSDok, understood10:19
ShilpaSDso can we not just add postgreSQL support in _validate_engine()?10:19
ShilpaSDas i have added sqlite for funtiobnal Test cases10:20
abhishekkcorrect10:21
abhishekkthat's not the right way10:21
ShilpaSD Actually calling _sync(), here migration scripts are not taking into account.10:23
ShilpaSDAlembic command upgrade, reads versions set inside scripts and execute the sequence10:23
ShilpaSDEarlier pike01/ocata01 files are consolidated files of EMC, and i have removed them in recent fix (https://review.openstack.org/#/c/433934/)10:24
abhishekkShilpaSD, I doubt that for offline migration we need to run any migration scripts10:24
abhishekkas it does not make any sense10:25
ShilpaSD please elaborate this for my understanding10:25
abhishekkShilpaSD, check the upgrades specs it is explained in their10:26
*** aniketh has joined #openstack-glance10:27
ShilpaSDyes10:28
ShilpaSDi want to understand why migration (M) is not important for offline10:29
*** alexchadin has quit IRC10:31
abhishekkShilpaSD, do you know why migration scripts are required?10:33
ShilpaSDyes10:33
abhishekkShilpaSD, tell me why10:34
ShilpaSDif we have any column altered, to sync data between old and new column, migrate scripts are required10:35
abhishekkyes but that is specifically in online case,10:36
abhishekkin case of offline migration you can directly execute the alter command as you don't need to copy data from one column to another column10:37
abhishekkyou can simply rename that column10:37
ShilpaSDtotally agree, but concern is earlier to support this there were two types scripts, pike01 and (pike_expand01/pike_migrate01/pikecontract01), and now going further, there won't be such scripts, so how we will provide operators such scripts which will consider alter case.10:41
ShilpaSDnow only EMC scripts willbe there*10:42
*** dtantsur is now known as dtantsur|bbl10:43
abhishekkShilpaSD, try to setup devstack with my patch, I can see visibility column in the images table of postgresql10:44
abhishekkif that was the case, there should be is_public column instead of visibility10:45
*** abhishekk is now known as abhishekk|brb10:46
jokke_abhishekk|brb: the migration scripts are executed and will need to be executed in offline as well10:48
abhishekk|brbjokke_, ok10:48
jokke_abhishekk|brb: lets say there is column moved to another table, the data does not just magically appear there when the column is created10:48
jokke_there is migration needed to sync the old and new column location before removing the old column10:49
abhishekk|brbjokke_, so in my case as I stated earlier I am able to see visibility column in images table, that means it runs those scripts as well10:49
abhishekk|brbjokke_, agree, that is required in case of moving the column to a new table10:50
jokke_it's not common for us to happen, but oversight of such case would cause horrific consequencies10:51
abhishekk|brbI have just created a new setup where I am running devstack again, from logs I will be able to confirm which scripts are executed10:51
jokke_cool10:51
abhishekk|brbjokke_, Strongly agree10:51
abhishekk|brbi will be back shortly10:52
jokke_sure10:52
jokke_abhishekk|brb: when you come back, wasn't that _sync(version) the same function we used before moving everything running E-M-C?10:53
*** ratailor has quit IRC10:54
*** d0ugal has quit IRC10:59
abhishekk|brbjokke_, yes, that's why I thought it will be easy fix10:59
*** abhishekk|brb is now known as abhishekk11:00
*** pbourke has quit IRC11:08
abhishekkjokke_, as far as I know previously for db sync it was running scripts from db/sqlalchemy/migrate_repo/versions directory11:10
abhishekkit has nothing to do with scripts provided in alembic_migrations repo11:11
*** d0ugal has joined #openstack-glance11:16
abhishekkjokke_, I have created a new setup and the db sync command executes scripts from db/sqlalchemy/migrate_repo/versions directory and then from db/sqlalchemy/alembic_migrations/versions table ( same was the case before)11:21
*** abhishekk has quit IRC11:24
*** AlexeyAbashkin has quit IRC11:27
*** alexchadin has joined #openstack-glance11:28
*** pdeore has quit IRC11:29
*** AlexeyAbashkin has joined #openstack-glance11:30
*** udesale has quit IRC11:35
bhagyashri_sjokke_: Hi, I just want to ask one thing related to hook-- is it possible to make the third party lib task configurable using hook?11:39
jokke_bhagyashri_s: what do you mean?11:44
bhagyashri_sjokke_: i mean task implementation is in third party lib and that i want to make it configurable and add that task in taskflow11:45
jokke_bhagyashri_s: if you look at 824badc1890206695eb335582a804d9ecabb4d6711:45
jokke_it was made to make it easier for 3rd party plugins to tap in11:46
*** nicolasbock has joined #openstack-glance11:47
jokke_bhagyashri_s: so if you're deploying plugin that is not provided by Glance release, you drop it to plugins folder and add it to plugin_opts.py in that folder and then you need to add it to glance.image_import.plugins in the setup.cfg11:49
jokke_so drop your .py file and add it to those two files and you should be able to enable it in the config11:50
jokke_if it defines any config opts configgen should pick them up and add them into the glance-image-import.conf.sample11:51
jokke_coffee ... brb11:52
bhagyashri_sbhagyashri_s: ok. will talk later11:54
jokke_back12:05
bhagyashri_sjokke_: in that case how the inputs will be provided to that tasks to execute it successfully12:06
jokke_bhagyashri_s: those deployer plugins _needs_ to be transparent for enduser, so you don't12:07
jokke_they will be in the pipeline just like the metadata injection, it might get config from the deployment's config files, it may use context but you never expect the person who makes the imput call to even know about your plugin12:08
bhagyashri_sjokke_: for example there is one Task say TaskA which is suppose not in glance code that is located in third party lib and that task i want to make it configurable and want to add it in taskflow12:09
bhagyashri_sok12:09
jokke_bhagyashri_s: well you would need to make some kind of wrapper for it into the plugins12:10
jokke_so if you use just random task that is not designed for these limitations, you would need to wrap it to something that can handle all that12:10
*** alexchadin has quit IRC12:11
*** alexchadin has joined #openstack-glance12:12
bhagyashri_sjokke_: ok. and is it possible we can give the class patch of task instead of module name of task under plugin folder in setup.cfg12:13
bhagyashri_sjokke_: or we can also say that giving the class name of task instead of module name in configuration to load that task12:16
*** alexchadin has quit IRC12:16
*** alexchadin has joined #openstack-glance12:17
jokke_bhagyashri_s: no12:17
bhagyashri_sjokke_: for example no_op = glance.async.flows.plugins.no_op:_Noop in setup.cfg12:17
bhagyashri_sjokke_: ??12:17
jokke_the plugin framework needs that task fully initialized from the get_task function12:17
jokke_so that it can be just dropped into the taskflow queue12:18
jokke_so you would need to make plugins/my_task_wrapper.py that handles all that and imports and initializes the task object and returns it like the example does12:19
jokke_so obviously if you have that task coming from some library you don't need to implement the task class in the my_task_wrapper.py, you just import it and implement that get_flow() function12:20
*** alexchadin has quit IRC12:20
*** alexchadin has joined #openstack-glance12:21
bhagyashri_sjokke_: is there any reference where i can see this12:21
jokke_and then you specify it like my_task = glance.async.flows.plugins.my_task_wrapper:get_flow12:21
jokke_bhagyashri_s: no12:22
bhagyashri_sjokke_: and plugin framework i want understand more so is there any link12:22
jokke_I'm working on it, but it does not currently exist and I'm unlikely going cover importing some random task and wrapping it12:22
jokke_the task framework we're using is taskflow12:23
bhagyashri_si want to more understand about plugin framework12:23
jokke_bhagyashri_s: the taskflow is bit like middleware in the wsgi environment ... it's pipeline of features ... taskflow takes care of that pipeline being executed without the user needing to sit there for whole time waiting12:25
bhagyashri_sin short setting class patch or making the task configurable using the class name is not doable directly get_flow is imp12:25
bhagyashri_sjokke_: ok12:25
jokke_and those plugins are like the wsgi middleware ... it's a task that gets added to the pipeline, it needs to understand the inputs and limitations what info it has available and pass that info through12:26
jokke_and what we do with the plugins, the no_op is kind of minimal what that task needs to handle so it can be executed12:27
jokke_the no_op does not do anything but it has all the piping in place so it can be dropped into that pipeline and it won't break it12:28
jokke_and the whole idea for those plugins is that, lets say cloud provider does not want a single image to be sent to nova before they have been checked by antivirus. They can implement plugin that does the viruscheck on the image data. The end user does not need to even know about this, their image import just takes bit longer12:29
jokke_so for the sake of interoperability those plugins must not expect any special input from the end user12:30
*** bhagyashri_s is now known as bbs12:31
jokke_so that the user can make the same import call to 10 different clouds and at the end will have 10 images ready as long as their image is within the constrains of what the cloud provider is accepting12:31
bbsjokke_: ok.12:32
*** awestin1 has quit IRC12:32
*** awestin1 has joined #openstack-glance12:33
*** dtantsur|bbl is now known as dtantsur12:40
bbsjokke_: thank you :)12:42
*** pooja-jadhav has quit IRC12:48
*** serverascode has quit IRC12:48
jokke_np12:48
jokke_I have talk proposed to Vancouver summit about the plugins12:49
*** serverascode has joined #openstack-glance12:49
jokke_so even if you can't be there in person for it, please vote it to be accepted as all of the talks will be recorded12:49
*** ameade has quit IRC12:50
*** ameade has joined #openstack-glance12:51
jokke_and I'll try to get written documentation for the Import plugins done as well ... there has been just so much firefighting on this cycle to be even able to get everything into the release we wanted that I haven't had time for it12:51
*** mosulica has quit IRC12:54
*** tesseract has quit IRC13:12
openstackgerritMerged openstack/glance master: Fix wrong usage of extend in list_image_import_opts  https://review.openstack.org/54269913:14
bbsjokke_: ok thank you :) for your inputs and sure i will vote13:16
bbsjokke_: actually my use case was to give the class patch to configure any task but it's difficult to do this13:17
*** konan has joined #openstack-glance13:27
konanjokke_: Abhishek here, sorry I was driving and didn't noticed you have also cherry-picked that patch to stable/queens13:28
jokke_konan: oh, you did too?13:29
jokke_no worries ... I was just following it and did the backport as soon as I saw the master merging13:30
konanYes, it has uploaded new ps to the same review13:30
konanWill take care next time13:30
jokke_konan: don't worry about it ... I was expecting you to be home already enjoying your well deserved weekend13:31
*** konan has quit IRC13:31
jokke_konan: I do hope you compensate these late evenings with late mornings as well13:31
*** konan has joined #openstack-glance13:32
konanjokke_: thank you :)13:32
jokke_The fact that I do poor work/life balance choices does not mean that others should do so too :P13:34
*** alexchadin has quit IRC13:39
anikethjokke_: Hi13:42
anikethjokke_: Just completed the whole setup to do my first commit, whole Gerrit, account setup and stuff.13:44
anikethCan you point me to some smaller task that I could try to fix and get acquainted with the codebase?13:44
*** alexchadin has joined #openstack-glance13:44
* aniketh reading the doc on Glance architecture. 13:45
*** alexchadin has quit IRC13:47
*** alexchadin has joined #openstack-glance13:49
*** alexchadin has quit IRC13:52
*** Supun has joined #openstack-glance13:52
*** edmondsw has joined #openstack-glance13:54
*** links has quit IRC13:56
*** alexchadin has joined #openstack-glance14:01
*** rosmaita has joined #openstack-glance14:03
openstackgerritMerged openstack/glance master: Imported Translations from Zanata  https://review.openstack.org/54264214:05
jokke_rosmaita: ping14:16
rosmaitajokke_ pong14:16
*** kairat has quit IRC14:18
jokke_rosmaita: when you have time, couple of things around rc214:20
jokke_let me know when you're caffeinated14:20
rosmaitai guess now is ok14:22
rosmaitai have a question about https://review.openstack.org/#/c/542814/2/glance/opts.py14:22
jokke_that was one of the reviews I wanted to flag14:22
jokke_so shoot14:22
rosmaitaif list_image_import_opts() is called more than once, isn't the list going to get longer and longer because of the extend?14:23
rosmaita i think you'll add the plugin opts over and over14:23
jokke_good point, it will and that was one of the reasons I wanted to have it in the local var, now I'm not sure if anything else than configgen actually calls that function14:24
rosmaitamabye we want _image_import_opts + get_plugin_opts14:25
jokke_but I'm more than willing to make combination of those two to ensure that continuous expansion does not happen14:25
rosmaitai'm thinking your first intuition was close14:25
rosmaitado something like opts = importopts + plugin_opts14:25
rosmaitaand then use that in the list comprehension in the return14:26
jokke_yeah, so lets rever the patch from master and redo it properly14:27
rosmaitaok, i can do that14:27
*** ShilpaSD has quit IRC14:28
jokke_then there is patch from Abhishek fixing our postgre issue ... quite well I'd say if I understand it correctly14:28
jokke_it just needs bunch of documentation around it14:28
rosmaitaoh man that is fantastic14:28
*** tesseract has joined #openstack-glance14:28
rosmaitaok, well i can work up some documentation while trying to understand his patch14:28
jokke_and the thrid one is the web-download I wanted to talk about14:29
jokke_so I just realized that we enabled it by default14:29
rosmaitaright14:29
rosmaitai was going to add a security note14:29
jokke_and it's still just feature parr with the v1 as in the security issue is still there14:29
rosmaitathat's what i was thinking14:30
rosmaitabut we could ship with it "off"14:30
rosmaitaeither way14:30
rosmaitabut we do need to bring it to operators attention14:30
rosmaitai guess the responsible thing is security note _+ "off" by default14:30
jokke_I'm working on plugging that, _but_ do we want to have that backported as bugfix for it, or is it a feature that needs to land in rocky in which case we probably should drop it from the defaults14:30
rosmaitahow complicated will it be to do?14:31
rosmaitaif you can get it done so we can release it as part of rc-2 on thursday14:31
rosmaitathen we have roughly a week to test14:31
jokke_not so ... urllib.parse decomposes the uri quite nicely, then it's just matter of comparison if those components are in black-/whitelists that needs to be introduced as config options14:32
jokke_lots of rows due to those configs and their helptexts but the logic is quite simple14:32
rosmaitai personally would like to see it in queens (of course)14:32
rosmaitalet's do it as a bugfix14:33
rosmaitaand if it gets complicated, we go with plan B14:33
jokke_cool, I have meeting with my manager at the top of the hour but I'll try to get it done after so you can have a look still today14:33
jokke_need to get the bug effectively copied and targeted to -rc2 then14:33
rosmaitaman, well if we have your and abhishek's stuff working, we can do rc-2 really early and have lots of testing time14:34
jokke_yes I'd like to get rc2 out Mon or Tue14:34
jokke_unless something else than the once we're currently aware of pops out14:34
jokke_ones14:34
jokke_and preferably that without working whole weekend to make it happen :P14:35
openstackgerritErno Kuvaja proposed openstack/glance master: Revert "Fix wrong usage of extend in list_image_import_opts"  https://review.openstack.org/54285514:38
*** stephenfin is now known as finucannot14:38
rosmaitajokke_ i was just about to revert https://review.openstack.org/#/c/542699/14:39
rosmaitaam i looking at the wrong review?14:39
rosmaitadumb me ... yours is the actual reversion14:41
openstackgerritErno Kuvaja proposed openstack/glance master: Fix bad usage of extend in list_image_import_opts  https://review.openstack.org/54286014:43
rosmaitai think deepcopy is a better choice than + there!14:44
*** prometheanfire has left #openstack-glance14:44
jokke_rosmaita: those two are revert and fix ^^14:44
rosmaitacool, should i ninja them in?14:44
jokke_bit less performant and memory hog but lots safer and those lists should never get big anyways14:45
rosmaitayeah, and pretty unlikely they'll be called more than a few times14:45
jokke_indeed14:45
jokke_my original patch was kind of hastely bad thinking of how that list extend works14:46
jokke_tried to compress it too much ... lines are cheap :P14:46
rosmaitawell, it looked good ... pretty sure both me and abhishek ok'd it14:46
jokke_yeah ... we all thought it wrong way ... fortunately it was quick fix, the second time we overlooked it :P14:47
jokke_*after the second time*14:47
jokke__really_ nice catch btw right off the bat at the morning :)14:48
rosmaitayeah, and only at 33% caffeination!14:49
rosmaitai am going to get up to 66% and then tackle abhishek's postgres patch14:49
jokke_cool, fresh coffee, meeting and then back to that uri filter patch14:49
jokke_I have it like half way done14:49
*** hogepodge has quit IRC14:51
*** hogepodge has joined #openstack-glance14:52
*** Shilpa has quit IRC14:53
*** bbs has quit IRC14:54
rosmaitacool, verified your change locally, as soon as those pass zuul check, i will ninja them in14:54
*** Supun has quit IRC14:55
*** Supun has joined #openstack-glance14:56
*** ykarel is now known as ykarel|away14:56
*** r-daneel has joined #openstack-glance14:57
*** ykarel|away has quit IRC15:01
*** Supun has quit IRC15:02
anikethrosmaita: Ping15:04
anikethCould we talk? Sorry for being late. Just time line issue :(15:04
rosmaitaaniketh how about at 15:30 utc?15:04
anikethrosmaita: Yes, That works perfectly fine :D15:05
rosmaitaolk15:05
*** alexchadin has quit IRC15:08
*** ratailor has joined #openstack-glance15:11
anikethrosmaita: Hey, you there?15:32
rosmaitahello15:32
anikethJust repeating what I said earlier today in the channel15:33
anikethJust completed the whole setup to do my first commit, whole Gerrit, account setup and stuff.15:33
anikethCan you point me to some smaller task that I could try to fix and get acquainted with the codebase?15:33
rosmaitasorry, i was offline and didn't read the scrollback15:34
rosmaitagive me a few min to locate something15:34
anikethYes, I know it is fine :)15:34
anikethAlso, what I'm now unaware of is how to reproduce a bug, that I chose to fix and if I made some changes in the code base regarding that - how could I test that, to verify if the fix is right or not? I guess this is because - I have cloned the repo of Glance into my localhost, I don't know how to run that or build it. Knowing this, would help the above I assume?15:36
anikethrosmaita: ^15:36
rosmaitaaniketh so the bug should have some description of how it happens15:37
rosmaitaat that point, you have 2 choices:15:37
anikethyes15:37
rosmaitareproduce it in your devstack15:37
rosmaitawrite a test that fails (that is, catches the bug) in your local repo15:38
rosmaitait's usually best to do both15:38
anikethok15:38
rosmaitareproducing in devstack is helpful so you can see what's going on15:38
rosmaitaand you'll probably be asked to have a test along with the bugfix anyway15:38
anikethYeah! -- oh, okay! Alright.15:39
rosmaitabut on your local machine, you should be able to do tox -e py27 to run the unit tests15:39
rosmaitaif you just do tox15:39
rosmaitait will run everything15:39
*** AlexeyAbashkin has quit IRC15:40
rosmaitaand you can also run just specific tests15:40
rosmaitaso that you don't have to wait forever to see what happens15:41
*** AlexeyAbashkin has joined #openstack-glance15:42
anikethtox -e py27 -- project_name_here/tests/folder_name1_here/folder_name2_here/test_file_name_here.py:TestClassName.test_method_name15:42
aniketh?15:42
rosmaitayeah, something like that ... i have to admit, i do it the hard way (activate the appropriate tox virtualenv and then run python -m testtools.run <fully-qualified-test-name>15:43
rosmaitaaniketh were you able to get devstack running?15:44
anikethrosmaita: Oh, which is better to be followed then?15:44
anikethrosmaita: Yes15:44
rosmaitaaniketh tox , if it works for you15:44
anikethokay! I haven't tried it yet. Will try and see while writing tests.15:45
*** AlexeyAbashkin has quit IRC15:46
*** Supun has joined #openstack-glance15:48
anikethrosmaita: So, after cloning the Glance repo in the local system, I don't have to build it or do anything else right?15:48
rosmaitaright, you should be able to run all the commands out of tox without actually building and/or installing glance15:49
anikethokay, alright :D15:50
rosmaita(still looking for something for you)15:50
anikethalso, btw, you mentioned about functional testing as well inside Glance itself. Could you help me with that?15:50
anikethrosmaita: Thanks :)15:51
rosmaitasure15:51
rosmaitawhen you look at the glance repo, you'll find glance/tests/functional15:51
rosmaitathat's where they live15:51
rosmaitayou can look at the base classes, they set up servers for the tests to interact with15:51
rosmaitayou run them with tox -e functional or tox -e functional-py3515:52
anikethoh okay.15:54
anikethBtw, all the tests to be written and which are already written exist in glance/tests/ right?15:55
rosmaitaright, they're organized into sub directories for unit, integration, and functional15:55
rosmaitaand then subdirectories under that for various components15:56
anikethYes - Neat!15:56
anikethI believe - if I get some small bug to fix, I could get acquainted with the codebase a bit as well as understand more of this structure in Glance even more and even better.15:59
*** r-daneel has quit IRC15:59
rosmaitayou are right about that!15:59
*** Supun has quit IRC16:00
aniketh:D16:01
anikethrosmaita: Also, if there is some documentation that I should be reading as a beginner or which I should be visiting quite often(Like devs manual), please do point to me that as well :D16:01
*** pcaruana has quit IRC16:01
rosmaitaaniketh i put something together about that for the last summit, i should be able to find it real quickly16:01
anikethRight now, I am going through the following16:01
anikethhttps://docs.openstack.org/glance/pike/contributor/index.html16:02
anikethhttps://docs.openstack.org/infra/manual/developers.html16:02
anikethhttps://docs.openstack.org/glance/latest/16:02
anikethrosmaita: oh, that would be great, If I can get my hand to it ;)16:02
rosmaitathose are all good16:04
rosmaitathis might have some good tips, too: https://www.openstack.org/videos/boston-2017/how-to-train-an-openstack-dragon16:04
*** ratailor has quit IRC16:04
*** e0ne has quit IRC16:06
* aniketh going through the video :D16:07
rosmaitathis has some info about contributing: https://etherpad.openstack.org/p/SYD-forum-glance-onboarding16:09
*** itlinux has joined #openstack-glance16:10
jokke_aniketh: also review lots of patches submitted to gerrit. That makes all the difference understanding how people approaches the issues and how the code is constructed16:16
anikethjokke_: Yes, Thanks for the suggestion. I was doing that earlier. Even though, I wasn't able to understand much :P16:19
anikethjokke_: I will continue again. :D16:19
*** konan has quit IRC16:21
anikethrosmaita: Thanks for the links and the video, will dig into those today and probably tomorrow (IST) as well.16:28
anikethrosmaita: Also, please do link me up with some bug which I could try on and help me with a confidence boost maybe :D16:28
rosmaitaaniketh : i'm still looking, got sidetracked while looking at some of these bugs16:29
anikethrosmaita: Oh, it is fine :D16:30
rosmaitaaniketh i was just about to close this one as "invalid", it would be helpful if you could confirm that it is not an issue: https://bugs.launchpad.net/glance/+bug/173350016:30
openstackLaunchpad bug 1733500 in Glance "image-show interface with Chinese characters return 500 Internal Server Error" [Undecided,Incomplete] - Assigned to weikeyou (weikeyou)16:30
rosmaitathere are some notes in there about how to test16:31
rosmaitayou can use curl and your devstack to see what happens16:31
anikethokay, So it basically asks to write test for it, I guess. I will see to it :D16:31
rosmaitayou know, we should have unit tests already that make sure utf-8 is handled correctly16:32
rosmaitayou could check to verify that16:32
rosmaitaand actually, here is a bug for you: https://bugs.launchpad.net/glance/+bug/173773816:33
openstackLaunchpad bug 1737738 in Glance "Unable to scrub image " [Undecided,New]16:33
rosmaitayou'll have to read up on the delayed-delete feature and how to turn it on in your devstack16:34
rosmaitaaniketh ^^16:35
anikethrosmaita: Yes, I wil take a look into both of those.16:36
rosmaitacool! thanks16:36
anikethThank you :D16:37
rosmaitaaniketh: is your launchpad id also 'aniketh' ?16:46
openstackgerritMatthew Treinish proposed openstack/glance master: Specify a python executable for prlimits when using uwsgi  https://review.openstack.org/53149816:49
*** links has joined #openstack-glance16:49
*** danpawlik has quit IRC16:59
*** Supun has joined #openstack-glance17:03
*** AlexeyAbashkin has joined #openstack-glance17:12
*** links has quit IRC17:15
*** konan has joined #openstack-glance17:18
*** konan is now known as abhishek17:29
abhishekjokke_, rosmaita just finished with your conversation about extend issue17:30
rosmaitaabhishek hope that made sense?17:30
abhishekYes, indeed17:31
rosmaitathe revert is in the gate now17:31
rosmaitaif you want to +2 https://review.openstack.org/#/c/542860/17:31
*** Supun has quit IRC17:31
rosmaitai will +2A it when the reversion is merged17:31
abhishekSaw that as well, some problem with my mail client (not receiving mails), so I logged in and read all the discussion17:32
abhishekrosmaita: definitely17:32
rosmaitaabhishek nice work on the postgres thing17:32
rosmaitai'll be looking more closely after lunch, but that's a big load off my mind17:32
rosmaitai was worried that it would be much uglier17:33
abhishekI am hoping this will solve our purpose,17:33
abhishekYes need some careful review17:33
jokke_abhishek: rosmaita: would you have time to help me out a bit?17:34
abhishekIf something is needed, drop a mail on konan.abhi@gmail.com I will look at them tomorrow morning time17:34
rosmaitaok, thanks17:34
abhishekjokke_: yes17:35
rosmaitajokke_ i will help, i think it's like 11pm where abhishek is17:35
jokke_ok, let me push the wip to gerrit, easier to explain17:35
rosmaitasounds good17:35
openstackgerritErno Kuvaja proposed openstack/glance master: WIP: Uri Filtering for web-download  https://review.openstack.org/54295617:35
rosmaitabut abhishek if you are not sleepy yet, by all means stick around17:35
rosmaita:)17:35
abhishekYes I will, just having my dinner17:36
jokke_images.py line 10117:36
jokke_at which point we parse that body? do I need to do it there and if so what should I call to get the needed info out of that body part?17:37
*** harlowja has joined #openstack-glance17:37
jokke_that line fails on py27 tests with None not having ['method']17:37
jokke_tried with .get() as well, which obviously failed :P17:38
abhishekbody is list or dict?17:40
jokke_it should be dict after it's parsed from json iirc17:41
jokke_but if I change it to .get("method") I get "NoneType" object has no attribute 'get'17:42
abhishekNeed to start my laptop, to have clear look17:46
*** abhishekk has joined #openstack-glance17:48
*** r-daneel has joined #openstack-glance17:48
*** AlexeyAbashkin has quit IRC17:48
rosmaitai'm looking at _validate_import_body(), we prob need to add some stuff there to make sure the web-download also has a uri in it (doesn't address your question, just something i'm noticing)17:49
jokke_rosmaita: it doesn't matter, really as the check I wrote already covers that case17:50
jokke_rosmaita: obviously if we do it in the deserialization phase, then we don't need to do it in validation phase17:51
rosmaitajokke_ what does the request look like?17:52
jokke_rosmaita: the request should have all the headers etc.17:52
abhishekkbody looks like, {u'method': {u'name': u'glance-direct'}}17:54
jokke_this would have been so much easier if I did it in the task, but that would be just wrong as then the import call would go through and the import task would just fail ... horribly17:54
jokke_abhishek: and with web-download it should look like {u'method': {u'name': u'web-download', u'uri': u'insert-uri-here'}}17:55
abhishekkyes17:55
openstackgerritMerged openstack/glance master: Revert "Fix wrong usage of extend in list_image_import_opts"  https://review.openstack.org/54285517:55
rosmaitatake a look at line 83117:55
abhishekkjust a minute let me reopen it17:56
rosmaitai think maybe there is one more layer of indirection?17:56
jokke_oh, gr8 ... that actually may be it, let me try17:57
abhishekki guess so17:57
rosmaitaso maybe import_method = body['body']['method']['name']17:57
*** Supun has joined #openstack-glance17:59
*** AlexeyAbashkin has joined #openstack-glance18:00
jokke_with body['body'] I get KeyError: 'body'18:00
rosmaitayuck18:00
jokke_just like with method I get KeyError: 'method'18:00
abhishekkbut then if import-method is glance-direct then it should show body like {'body': {u'method': {u'name': u'glance-direct'}}}, but its not18:01
rosmaitamaybe put in a print("%r" % dir(body)) and let's see what it is18:04
rosmaitabecause i for one am confused18:04
*** AlexeyAbashkin has quit IRC18:04
abhishekkjust a minute18:08
*** AlexeyAbashkin has joined #openstack-glance18:08
abhishekkrosmaita, ['__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'clear', 'copy', 'fromkeys', 'get', 'has_key', 'items', 'iteritems',18:13
abhishekk 'iterkeys', 'itervalues', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values', 'viewitems', 'viewkeys', 'viewvalues']18:13
jokke_so it looks like dict18:14
abhishekkyes, its a dict18:15
rosmaitai guess print body.keys() and see what we get18:16
abhishekkgive me 5-10 minutes, network speed is suddenly down here18:22
anikethrosmaita: Sorry for the late reply. Yes, my launchpad id is aniketh0118:22
rosmaitaaniketh np,  thanks18:23
abhishekkErno that test is failing because we are passing empty dictionary from the test18:34
abhishekkjokke_,18:34
abhishekkjokke_, https://github.com/openstack/glance/blob/master/glance/tests/unit/v2/test_images_resource.py#L63818:35
jokke_abhishekk: I was looking, that it might be the case, but that just means that our deserializer is actually not running the validation ofr it18:35
jokke_it should have thrown bad request already from the deserializer18:36
abhishekkthe reason is from test we are calling direct controller's import_image method18:37
jokke_aargh18:37
abhishekkit doesn't call deserializer from tests18:37
rosmaitaarrgh18:37
rosmaitawhat jokke_ said18:37
jokke__This_ is why I hate unittests! :P18:37
abhishekkso instead of empty dictionary now you need to pass actual body from their18:38
jokke_abhishekk: I'm not any happier that it's Friday past midnight for you and you're still working on this (well I'm happy that you figured it out, but feel super bad about it at the same time)18:39
abhishekkjokke_, no issues18:39
abhishekkso would you like me to fix those tests?18:40
jokke_abhishekk: so I just add body={'method': {'name': 'glance-direct'}} as attribute to that get_fake_request?18:41
jokke_and that should sort us out?18:41
abhishekkjust give me a minute18:41
* rosmaita is being no help at all and is going to get lunch18:42
abhishekkjokke_, unfortunately that will not help, you need to replace {} with {'method': {'name': 'glance-direct'}}18:43
abhishekk5 tests from test_image_resources.py18:44
jokke_abhishekk: ohh, you mean in the asset()?18:44
abhishekkyes18:44
abhishekktest_image_import_raises_conflict_for_invalid_status_change18:45
jokke_cool ... thanks buddy, will do that right now and see how it behaves18:45
*** dtantsur is now known as dtantsur|afk18:45
abhishekktest_image_import_not_allowed18:45
abhishekktest_image_import_raises_conflict18:45
abhishekktest_image_import_raises_bad_request18:46
abhishekkand test_image_import18:46
abhishekklet me know if any other failures are their18:46
abhishekkafter this change py27 and py35 tests will pass18:49
jokke_it worked!18:50
abhishekkjokke_, I will be around for next 30 minutes, please let me know if anything is required18:50
abhishekkgreat18:50
jokke_you, my friend, are testing rockstar :D18:50
abhishekk:D18:50
abhishekkAnyway, I should have figured it out earlier, but I was under impression that something is wrong with the actual code18:51
jokke_because I was looking that we are throwing empty fake request at it, but didn't realize it bypassed the deserializer18:51
abhishekkwe have separate class to test deserializer part18:52
jokke_ok, let me put a commit message to it and it just might be good to go :D18:52
abhishekkgreat, I will have a look at entire patch tomorrow18:52
openstackgerritErno Kuvaja proposed openstack/glance master: URI filtering for web-download  https://review.openstack.org/54295618:57
jokke_gr8 thanks again abhishekk18:57
abhishekkjokke_, its pleasure, I will take your leave now18:59
abhishekkrosmaita, if you have any doubts/suggestion for postgresql patch, drop me a mail on konan.abhi@gmail.com (I will check tomorrow)19:00
*** abhishekk has quit IRC19:00
openstackgerritErno Kuvaja proposed openstack/glance master: URI filtering for web-download  https://review.openstack.org/54295619:14
jokke_rosmaita: abhishek ^^ works properly with configgen as well after the 542860 gets merged19:15
*** lbragstad has quit IRC19:18
rosmaitagreat!19:20
jokke_that should merge any second now and I'll propose the backport for it19:20
*** AlexeyAbashkin has quit IRC19:21
rosmaitacool19:21
openstackgerritMerged openstack/glance master: Fix bad usage of extend in list_image_import_opts  https://review.openstack.org/54286019:21
jokke_there we go :P19:21
rosmaita!!!19:21
openstackrosmaita: Error: "!!" is not a valid command.19:21
abhishekGreat19:23
jokke_rosmaita:  https://review.openstack.org/54298519:24
openstackgerritErno Kuvaja proposed openstack/glance master: URI filtering for web-download  https://review.openstack.org/54295619:29
*** idlemind has quit IRC19:37
*** idlemind has joined #openstack-glance19:38
*** AlexeyAbashkin has joined #openstack-glance19:49
*** abhishek has quit IRC19:49
*** AlexeyAbashkin has quit IRC19:53
*** tesseract has quit IRC20:00
*** dalgaaf has quit IRC20:04
*** Supun has quit IRC20:04
*** dalgaaf has joined #openstack-glance20:05
*** e0ne has joined #openstack-glance20:18
*** e0ne has quit IRC20:20
*** e0ne has joined #openstack-glance20:47
*** e0ne has quit IRC21:13
*** DuncanT has quit IRC21:49
*** DuncanT has joined #openstack-glance21:49
*** r-daneel has quit IRC22:09
*** aniketh has quit IRC22:14
openstackgerritBrian Rosmaita proposed openstack/glance master: Use configured value for import-methods header  https://review.openstack.org/54303922:20
openstackgerritErno Kuvaja proposed openstack/glance master: URI filtering for web-download  https://review.openstack.org/54295622:37
jokke_rosmaita: Abhishek's postgre fix worked ... the Manila postgre tests passed :D22:44
jokke_makes me happy ... how much couple of days of working gate makes for productivity :D22:44
*** cargonza has quit IRC22:53
*** cargonza has joined #openstack-glance22:54
*** edmondsw has quit IRC22:59
*** esberglu has quit IRC23:37

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!