| *** zul has quit IRC | 00:01 | |
| *** bhagyashri has quit IRC | 00:11 | |
| *** ducttape_ has quit IRC | 00:12 | |
| *** mingyu has joined #openstack-nova | 00:14 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova: Fix policy tests for project_id enforcement https://review.openstack.org/354690 | 00:18 |
|---|---|---|
| *** felix23ma has joined #openstack-nova | 00:19 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova: Fix policy tests for project_id enforcement https://review.openstack.org/354690 | 00:20 |
| *** ducttape_ has joined #openstack-nova | 00:20 | |
| *** liangy has quit IRC | 00:21 | |
| *** Apoorva_ has joined #openstack-nova | 00:24 | |
| *** trisq has joined #openstack-nova | 00:26 | |
| *** Apoorva has quit IRC | 00:27 | |
| *** claudiub|2 has quit IRC | 00:28 | |
| *** Apoorva_ has quit IRC | 00:28 | |
| *** harlowja has joined #openstack-nova | 00:29 | |
| *** ducttape_ has quit IRC | 00:30 | |
| *** diogogmt has joined #openstack-nova | 00:35 | |
| *** akshai has joined #openstack-nova | 00:39 | |
| *** kaisers1 has joined #openstack-nova | 00:42 | |
| *** ducttape_ has joined #openstack-nova | 00:43 | |
| *** kaisers has quit IRC | 00:43 | |
| *** thorst has joined #openstack-nova | 00:43 | |
| *** hoangcx has joined #openstack-nova | 00:46 | |
| *** akshai has quit IRC | 00:48 | |
| *** eharney has quit IRC | 00:49 | |
| *** cdelatte has quit IRC | 00:51 | |
| *** gouthamr_ has quit IRC | 00:51 | |
| *** kevinz has joined #openstack-nova | 00:52 | |
| *** jdurgin1 has joined #openstack-nova | 00:55 | |
| *** mingyu has quit IRC | 00:56 | |
| *** amrith has left #openstack-nova | 00:56 | |
| *** kevinz has quit IRC | 00:57 | |
| *** gongysh has joined #openstack-nova | 00:58 | |
| *** ssurana1 has quit IRC | 00:58 | |
| *** dimtruck is now known as zz_dimtruck | 01:00 | |
| *** unicell has quit IRC | 01:00 | |
| *** rfolco has quit IRC | 01:02 | |
| *** akshai has joined #openstack-nova | 01:03 | |
| *** trisq has quit IRC | 01:03 | |
| *** ducttape_ has quit IRC | 01:03 | |
| openstackgerrit | Eli Qiao proposed openstack/nova: Libvirt: Correct PERF_EVENTS_CPU_FLAG_MAPPING https://review.openstack.org/365842 | 01:05 |
| *** zz_dimtruck is now known as dimtruck | 01:06 | |
| *** gouthamr has joined #openstack-nova | 01:06 | |
| *** mars_toktonaliev has joined #openstack-nova | 01:08 | |
| *** cheneydc has joined #openstack-nova | 01:09 | |
| *** cheneydc has quit IRC | 01:10 | |
| *** Sukhdev_ has quit IRC | 01:10 | |
| *** marst has quit IRC | 01:12 | |
| *** raj_singh_ has joined #openstack-nova | 01:13 | |
| suxiang | I have a question about security group of nova version juno.And i config security_group_api=neutron in the nova.conf.After creating a tenant and a new user belongs to the project,I create a VM on dashboard by using eth user.Then default security group was created in neutron.What makes me confused is that default security group is also stored in table security_groups of nova without uuid. | 01:13 |
| *** ducttape_ has joined #openstack-nova | 01:14 | |
| suxiang | After scanning source code of nova, I can't find where the action of storing default secrutiy group in table security_groups of nova happened.Can anybody tell me where it is? | 01:18 |
| *** READ10 has joined #openstack-nova | 01:22 | |
| *** salv-orlando has joined #openstack-nova | 01:23 | |
| *** raunak has quit IRC | 01:25 | |
| *** salv-orlando has quit IRC | 01:26 | |
| *** ducttape_ has quit IRC | 01:28 | |
| *** trisq has joined #openstack-nova | 01:29 | |
| *** xinliang has quit IRC | 01:30 | |
| *** wuhg has joined #openstack-nova | 01:31 | |
| *** sdake has joined #openstack-nova | 01:32 | |
| *** wangqun has joined #openstack-nova | 01:35 | |
| *** ducttape_ has joined #openstack-nova | 01:36 | |
| *** armax has quit IRC | 01:36 | |
| *** xinliang has joined #openstack-nova | 01:38 | |
| *** openstack has joined #openstack-nova | 01:44 | |
| *** vladikr has quit IRC | 01:44 | |
| *** gouthamr has quit IRC | 01:48 | |
| *** gouthamr has joined #openstack-nova | 01:49 | |
| *** liuyulong|away is now known as liuyulong | 01:49 | |
| *** baoli has joined #openstack-nova | 01:50 | |
| *** baoli has quit IRC | 02:08 | |
| *** bkopilov has quit IRC | 02:10 | |
| *** bkopilov_ has quit IRC | 02:10 | |
| *** tuanla_ has joined #openstack-nova | 02:12 | |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Enable py3 tests for unit.api.openstack.compute.test_console_output https://review.openstack.org/316678 | 02:13 |
| *** dimtruck is now known as zz_dimtruck | 02:14 | |
| *** hoangcx has quit IRC | 02:16 | |
| *** raunak has joined #openstack-nova | 02:16 | |
| *** ociuhandu has joined #openstack-nova | 02:18 | |
| *** hoangcx has joined #openstack-nova | 02:19 | |
| *** yamahata has quit IRC | 02:21 | |
| *** Swami has quit IRC | 02:25 | |
| *** ociuhandu has quit IRC | 02:26 | |
| gongysh | alex_xu, hi | 02:29 |
| alex_xu | gongysh: hi | 02:29 |
| gongysh | hi | 02:30 |
| *** ociuhandu has joined #openstack-nova | 02:30 | |
| gongysh | nova/policy.json is empty | 02:30 |
| gongysh | how does it do authz? | 02:30 |
| alex_xu | yea, all of those move to central place, nova/policies | 02:30 |
| *** READ10 has quit IRC | 02:31 | |
| gongysh | alex_xu, where? the keystone? | 02:31 |
| gongysh | alex_xu, it gets it from keystone? | 02:31 |
| alex_xu | gongysh: nothing changed for keystone | 02:31 |
| alex_xu | gongysh: just move the default policy rule into the python code | 02:31 |
| *** ducttape_ has quit IRC | 02:31 | |
| *** jdurgin1 has quit IRC | 02:32 | |
| *** ducttape_ has joined #openstack-nova | 02:32 | |
| alex_xu | gongysh: there is a tools you can generate the policy.json sample file just like oslo.config | 02:32 |
| *** salv-orlando has joined #openstack-nova | 02:33 | |
| alex_xu | gongysh: http://specs.openstack.org/openstack/nova-specs/specs/newton/approved/policy-in-code.html | 02:34 |
| *** ociuhandu has quit IRC | 02:34 | |
| *** ducttape_ has quit IRC | 02:34 | |
| *** armax has joined #openstack-nova | 02:34 | |
| *** baoli has joined #openstack-nova | 02:37 | |
| *** vishwana_ is now known as vishwanathj_zzz | 02:38 | |
| gongysh | alex_xu, why does not nova provide one place to do authz checking? | 02:41 |
| gongysh | alex_xu, the context.can is everywhere. | 02:41 |
| alex_xu | gongysh: need mapping to the each url? | 02:41 |
| gongysh | alex_xu, some kind of it. | 02:42 |
| alex_xu | gongysh: there is target in the context.can parameter | 02:42 |
| *** raunak has quit IRC | 02:42 | |
| alex_xu | you need get the target from the db | 02:42 |
| alex_xu | otherwise, as you said, just enforce the policy in one place | 02:43 |
| gongysh | alex_xu, in neutron, there is a layer after the API URL mapping, to do the authz. | 02:43 |
| *** mingyu has joined #openstack-nova | 02:43 | |
| *** salv-orlando has quit IRC | 02:43 | |
| alex_xu | gongysh: yea, but the code structure different with neutron | 02:43 |
| gongysh | alex_xu, there is no policy checking in other codes. | 02:43 |
| *** raunak has joined #openstack-nova | 02:44 | |
| *** raunak has quit IRC | 02:45 | |
| *** felix23ma has quit IRC | 02:47 | |
| *** tjones has joined #openstack-nova | 02:48 | |
| *** tjones has left #openstack-nova | 02:50 | |
| *** hoangcx has quit IRC | 02:51 | |
| *** hoangcx has joined #openstack-nova | 02:54 | |
| *** raunak has joined #openstack-nova | 02:54 | |
| gongysh | alex_xu, hi | 02:56 |
| alex_xu | gongysh: yea | 02:56 |
| gongysh | alex_xu. does normal user create flavor? | 02:56 |
| alex_xu | gongysh: no | 02:56 |
| *** ducttape_ has joined #openstack-nova | 02:56 | |
| *** ociuhandu has joined #openstack-nova | 03:14 | |
| *** raunak has quit IRC | 03:19 | |
| *** ociuhandu has quit IRC | 03:19 | |
| *** hoangcx has quit IRC | 03:27 | |
| *** bkopilov_ has joined #openstack-nova | 03:28 | |
| *** bkopilov has joined #openstack-nova | 03:28 | |
| *** ociuhandu has joined #openstack-nova | 03:29 | |
| *** mingyu has quit IRC | 03:32 | |
| *** mingyu has joined #openstack-nova | 03:32 | |
| *** ociuhandu has quit IRC | 03:34 | |
| *** hoangcx has joined #openstack-nova | 03:35 | |
| *** ducttape_ has quit IRC | 03:35 | |
| *** tongli has quit IRC | 03:38 | |
| *** raunak has joined #openstack-nova | 03:40 | |
| *** salv-orlando has joined #openstack-nova | 03:42 | |
| *** rfolco has joined #openstack-nova | 03:44 | |
| *** bhagyashri has joined #openstack-nova | 03:47 | |
| *** rfolco has quit IRC | 03:47 | |
| *** salv-orlando has quit IRC | 03:49 | |
| *** thorst has quit IRC | 03:53 | |
| *** thorst has joined #openstack-nova | 03:53 | |
| *** thorst has quit IRC | 03:58 | |
| *** thorst has joined #openstack-nova | 03:59 | |
| *** jgriffith is now known as jgriffith_away | 04:00 | |
| *** baoli has quit IRC | 04:01 | |
| *** ociuhandu has joined #openstack-nova | 04:03 | |
| *** lpetrut has joined #openstack-nova | 04:03 | |
| *** thorst has quit IRC | 04:04 | |
| *** GB21 has joined #openstack-nova | 04:05 | |
| *** ssurana has joined #openstack-nova | 04:06 | |
| *** sudipto has joined #openstack-nova | 04:07 | |
| *** sudipto_ has joined #openstack-nova | 04:07 | |
| *** ociuhandu has quit IRC | 04:08 | |
| *** thorst has joined #openstack-nova | 04:15 | |
| *** code-R has joined #openstack-nova | 04:19 | |
| *** rfolco has joined #openstack-nova | 04:20 | |
| *** RuiChen has quit IRC | 04:21 | |
| *** amotoki has joined #openstack-nova | 04:23 | |
| *** oanson has joined #openstack-nova | 04:23 | |
| *** links has joined #openstack-nova | 04:28 | |
| *** sudipto has quit IRC | 04:29 | |
| *** sudipto_ has quit IRC | 04:29 | |
| *** harlowja_at_home has joined #openstack-nova | 04:29 | |
| *** lpetrut has quit IRC | 04:30 | |
| *** markvoelker has quit IRC | 04:30 | |
| *** thorst has quit IRC | 04:30 | |
| *** markvoelker has joined #openstack-nova | 04:31 | |
| *** ekuris has joined #openstack-nova | 04:32 | |
| *** prateek has joined #openstack-nova | 04:32 | |
| *** prateek_ has joined #openstack-nova | 04:32 | |
| *** thorst has joined #openstack-nova | 04:33 | |
| *** hieulq has quit IRC | 04:35 | |
| *** GB21 has quit IRC | 04:35 | |
| *** hieulq has joined #openstack-nova | 04:35 | |
| *** ducttape_ has joined #openstack-nova | 04:36 | |
| *** prateek has quit IRC | 04:36 | |
| *** irenab has quit IRC | 04:37 | |
| *** shangxdy has joined #openstack-nova | 04:37 | |
| *** thorst has quit IRC | 04:38 | |
| *** diga has joined #openstack-nova | 04:38 | |
| *** dikonoor has joined #openstack-nova | 04:39 | |
| *** woodster_ has quit IRC | 04:39 | |
| *** can8dnSix has joined #openstack-nova | 04:40 | |
| *** ducttape_ has quit IRC | 04:40 | |
| *** fragatina has joined #openstack-nova | 04:41 | |
| *** fragatina has quit IRC | 04:45 | |
| *** GB21 has joined #openstack-nova | 04:51 | |
| *** salv-orlando has joined #openstack-nova | 04:52 | |
| *** gongysh has quit IRC | 04:52 | |
| *** dikonoor has quit IRC | 04:54 | |
| *** mfedosin has joined #openstack-nova | 04:55 | |
| *** dikonoor has joined #openstack-nova | 04:56 | |
| *** shangxdy has quit IRC | 04:56 | |
| *** sdake has quit IRC | 04:56 | |
| *** code-R_ has joined #openstack-nova | 04:57 | |
| *** akshai has quit IRC | 04:58 | |
| *** moshele has joined #openstack-nova | 04:58 | |
| *** salv-orl_ has joined #openstack-nova | 04:59 | |
| *** salv-orl_ has quit IRC | 04:59 | |
| *** salv-orlando has quit IRC | 04:59 | |
| *** mingyu has quit IRC | 04:59 | |
| *** salv-orlando has joined #openstack-nova | 05:00 | |
| *** sdake has joined #openstack-nova | 05:00 | |
| *** code-R has quit IRC | 05:00 | |
| *** Jeffrey4l__ has joined #openstack-nova | 05:00 | |
| *** lixiaoy1 has joined #openstack-nova | 05:01 | |
| *** Jeffrey4l_ has quit IRC | 05:03 | |
| *** sudipto has joined #openstack-nova | 05:06 | |
| *** sudipto_ has joined #openstack-nova | 05:06 | |
| *** thorst has joined #openstack-nova | 05:07 | |
| *** code-R_ has quit IRC | 05:08 | |
| *** thorst has quit IRC | 05:09 | |
| *** can8dnSix has quit IRC | 05:10 | |
| *** jaosorior has joined #openstack-nova | 05:10 | |
| *** unicell has joined #openstack-nova | 05:13 | |
| *** GB21 has quit IRC | 05:14 | |
| *** mfedosin has quit IRC | 05:15 | |
| *** ratailor has joined #openstack-nova | 05:15 | |
| *** unicell has quit IRC | 05:16 | |
| *** unicell has joined #openstack-nova | 05:16 | |
| *** nkrinner_afk is now known as nkrinner | 05:16 | |
| *** Shashi has joined #openstack-nova | 05:19 | |
| *** unicell1 has joined #openstack-nova | 05:22 | |
| *** unicell has quit IRC | 05:23 | |
| *** GB21 has joined #openstack-nova | 05:27 | |
| *** phuongnh has joined #openstack-nova | 05:29 | |
| *** links has quit IRC | 05:31 | |
| *** rfolco has quit IRC | 05:32 | |
| *** raj_singh_ has quit IRC | 05:32 | |
| *** mamandle has joined #openstack-nova | 05:33 | |
| *** dikonoor has quit IRC | 05:33 | |
| *** PaulMurray has joined #openstack-nova | 05:33 | |
| *** ducttape_ has joined #openstack-nova | 05:37 | |
| *** John341 has joined #openstack-nova | 05:40 | |
| *** ducttape_ has quit IRC | 05:41 | |
| John341 | Hi, I'm not sure if this is a bug or not: Why does 'nova quota-show --tenant ID' give different number of volumes, than 'openstack quota show ID', this causes my newly created projects and users not being able to provision VMs? | 05:42 |
| *** links has joined #openstack-nova | 05:46 | |
| *** harlowja_at_home has quit IRC | 05:46 | |
| *** code-R has joined #openstack-nova | 05:47 | |
| *** dikonoor has joined #openstack-nova | 05:48 | |
| *** lpetrut has joined #openstack-nova | 05:50 | |
| *** raj_singh_ has joined #openstack-nova | 05:50 | |
| *** gcb has joined #openstack-nova | 05:52 | |
| *** fragatina has joined #openstack-nova | 05:52 | |
| *** code-R_ has joined #openstack-nova | 05:55 | |
| *** raj_singh_ has quit IRC | 05:55 | |
| *** gouthamr has quit IRC | 05:55 | |
| *** sdake_ has joined #openstack-nova | 05:56 | |
| *** fragatina has quit IRC | 05:56 | |
| gcb | alex_xu, can you look at a simple patch to remove deprecated config option , it's nice to have it done in Newton https://review.openstack.org/319543 | 05:56 |
| *** gouthamr has joined #openstack-nova | 05:56 | |
| *** thorst has joined #openstack-nova | 05:57 | |
| *** code-R has quit IRC | 05:57 | |
| *** sdake has quit IRC | 05:58 | |
| *** thorst_ has joined #openstack-nova | 05:58 | |
| *** code-R_ has quit IRC | 06:00 | |
| *** thorst_ has quit IRC | 06:00 | |
| *** thorst has quit IRC | 06:01 | |
| *** dikonoor has quit IRC | 06:03 | |
| *** thorst has joined #openstack-nova | 06:03 | |
| *** annp has joined #openstack-nova | 06:04 | |
| *** paul-carlton2 has joined #openstack-nova | 06:07 | |
| *** PaulMurray has quit IRC | 06:07 | |
| Shashi | Hello, I created devsatck and configured backend sucessfully. Now when I am trying to create an Instance, I am gettinng error as "Instance failed network setup after 1 attempt" | 06:09 |
| Shashi | Can anyone help me out here? | 06:10 |
| *** unicell has joined #openstack-nova | 06:10 | |
| *** unicell1 has quit IRC | 06:13 | |
| *** baoli has joined #openstack-nova | 06:13 | |
| *** baoli has quit IRC | 06:17 | |
| *** gongysh has joined #openstack-nova | 06:20 | |
| *** liverpooler has quit IRC | 06:21 | |
| *** mamandle has quit IRC | 06:21 | |
| *** unicell1 has joined #openstack-nova | 06:24 | |
| *** unicell has quit IRC | 06:24 | |
| *** hoonetorg has quit IRC | 06:25 | |
| *** mjura has joined #openstack-nova | 06:25 | |
| *** yamahata has joined #openstack-nova | 06:26 | |
| *** ttrumm_ has joined #openstack-nova | 06:27 | |
| *** gongysh has quit IRC | 06:37 | |
| *** ducttape_ has joined #openstack-nova | 06:38 | |
| *** mingyu has joined #openstack-nova | 06:39 | |
| gmann | johnthetubaguy: alex_xu can you check this - https://review.openstack.org/#/c/354690/ | 06:39 |
| gmann | i remove the policy overridden tests | 06:39 |
| openstackgerrit | Ha Van Tu proposed openstack/nova: [api-ref] Correct parameter type https://review.openstack.org/367117 | 06:41 |
| openstackgerrit | huan proposed openstack/nova: xenapi: support the hotplug of a neutron port https://review.openstack.org/365505 | 06:41 |
| *** mamandle has joined #openstack-nova | 06:41 | |
| *** ducttape_ has quit IRC | 06:43 | |
| *** gouthamr_ has joined #openstack-nova | 06:47 | |
| *** tonytan4ever has quit IRC | 06:49 | |
| *** zenoway has joined #openstack-nova | 06:49 | |
| *** gouthamr has quit IRC | 06:51 | |
| *** nkrinner has quit IRC | 06:52 | |
| *** mamandle has quit IRC | 06:52 | |
| *** ttx has quit IRC | 06:54 | |
| *** salv-orlando has quit IRC | 06:54 | |
| *** sahid has joined #openstack-nova | 06:54 | |
| *** ttx has joined #openstack-nova | 06:54 | |
| *** hoangcx_ has joined #openstack-nova | 06:54 | |
| *** ttx has quit IRC | 06:54 | |
| *** ttx has joined #openstack-nova | 06:55 | |
| *** nkrinner has joined #openstack-nova | 06:55 | |
| *** hoangcx has quit IRC | 06:57 | |
| *** Jeffrey4l__ has quit IRC | 07:00 | |
| *** suxiang has quit IRC | 07:00 | |
| alex_xu | gmann: in a meeting, will check after meeting | 07:00 |
| *** claudiub|2 has joined #openstack-nova | 07:01 | |
| *** trisq has quit IRC | 07:01 | |
| *** ttrumm_ has quit IRC | 07:01 | |
| *** ssurana has quit IRC | 07:01 | |
| *** trisq has joined #openstack-nova | 07:02 | |
| *** nkrinner has quit IRC | 07:02 | |
| *** Jeffrey4l__ has joined #openstack-nova | 07:02 | |
| *** tesseract- has joined #openstack-nova | 07:04 | |
| *** matrohon has joined #openstack-nova | 07:08 | |
| *** trisq has quit IRC | 07:08 | |
| *** ttrumm has joined #openstack-nova | 07:09 | |
| *** trisq has joined #openstack-nova | 07:09 | |
| *** ttrumm_ has joined #openstack-nova | 07:10 | |
| *** ttrumm has quit IRC | 07:13 | |
| *** trisq has quit IRC | 07:14 | |
| *** trisq has joined #openstack-nova | 07:15 | |
| *** nkrinner has joined #openstack-nova | 07:15 | |
| *** roeyc has joined #openstack-nova | 07:17 | |
| *** sridharg has joined #openstack-nova | 07:17 | |
| *** oomichi has quit IRC | 07:18 | |
| *** oomichi has joined #openstack-nova | 07:19 | |
| *** adisky has joined #openstack-nova | 07:21 | |
| *** lixiaoy1 has quit IRC | 07:21 | |
| *** liverpooler has joined #openstack-nova | 07:22 | |
| openstackgerrit | Zhenyu Zheng proposed openstack/nova: Should not be able to sort instances based on joined tables https://review.openstack.org/359134 | 07:22 |
| *** jlanoux has joined #openstack-nova | 07:24 | |
| *** maestropandy has joined #openstack-nova | 07:25 | |
| *** maestropandy has left #openstack-nova | 07:25 | |
| *** irenab has joined #openstack-nova | 07:28 | |
| *** armax has quit IRC | 07:29 | |
| *** lixiaoy1 has joined #openstack-nova | 07:31 | |
| *** ttrumm has joined #openstack-nova | 07:32 | |
| *** jed56 has joined #openstack-nova | 07:33 | |
| *** openstackgerrit has quit IRC | 07:33 | |
| *** openstackgerrit has joined #openstack-nova | 07:34 | |
| *** gouthamr_ has quit IRC | 07:35 | |
| *** raunak has quit IRC | 07:35 | |
| *** ttrumm_ has quit IRC | 07:35 | |
| *** RuiChen has joined #openstack-nova | 07:36 | |
| *** thorst has quit IRC | 07:36 | |
| *** ttrumm has quit IRC | 07:37 | |
| *** markus_z has joined #openstack-nova | 07:37 | |
| *** ihrachys has joined #openstack-nova | 07:39 | |
| *** nherciu has joined #openstack-nova | 07:40 | |
| *** hoangcx_ has quit IRC | 07:43 | |
| *** ihrachys has quit IRC | 07:45 | |
| *** tonytan4ever has joined #openstack-nova | 07:50 | |
| *** ttrumm has joined #openstack-nova | 07:51 | |
| *** edand has joined #openstack-nova | 07:52 | |
| gmann | alex_xu: Thanks | 07:53 |
| *** tonytan4ever has quit IRC | 07:54 | |
| *** andreas_s has joined #openstack-nova | 07:56 | |
| *** ttrumm has quit IRC | 07:56 | |
| *** karthiks has joined #openstack-nova | 07:57 | |
| *** hoangcx has joined #openstack-nova | 07:57 | |
| *** danpb has joined #openstack-nova | 07:58 | |
| *** mingyu has quit IRC | 07:59 | |
| *** salv-orlando has joined #openstack-nova | 07:59 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** zzzeek has joined #openstack-nova | 08:00 | |
| *** maestropandy has joined #openstack-nova | 08:00 | |
| *** yamahata has quit IRC | 08:01 | |
| *** salv-orlando has quit IRC | 08:01 | |
| *** paul-carlton2 has quit IRC | 08:01 | |
| *** ttrumm has joined #openstack-nova | 08:02 | |
| *** paul-carlton2 has joined #openstack-nova | 08:02 | |
| *** ttrumm_ has joined #openstack-nova | 08:03 | |
| *** ttrumm_ has quit IRC | 08:05 | |
| *** ttrumm has quit IRC | 08:05 | |
| *** paul-carlton2 has quit IRC | 08:06 | |
| * johnthetubaguy returns from unpacking boxes | 08:07 | |
| *** kaisers has joined #openstack-nova | 08:12 | |
| *** murali__ has quit IRC | 08:13 | |
| *** maestropandy has left #openstack-nova | 08:15 | |
| *** ajo_ has quit IRC | 08:16 | |
| *** sdake_ is now known as sdake | 08:17 | |
| *** sridhargaddam has joined #openstack-nova | 08:18 | |
| *** jogo has quit IRC | 08:20 | |
| *** Jeffrey4l__ has quit IRC | 08:20 | |
| *** ttrumm has joined #openstack-nova | 08:20 | |
| *** sridharg has quit IRC | 08:21 | |
| *** yassine has joined #openstack-nova | 08:23 | |
| *** salv-orlando has joined #openstack-nova | 08:23 | |
| *** yassine is now known as Guest52068 | 08:23 | |
| *** dikonoor has joined #openstack-nova | 08:24 | |
| *** jogo has joined #openstack-nova | 08:25 | |
| *** paul-carlton2 has joined #openstack-nova | 08:25 | |
| *** salv-orl_ has joined #openstack-nova | 08:30 | |
| *** ihrachys has joined #openstack-nova | 08:31 | |
| *** salv-orlando has quit IRC | 08:33 | |
| wangqun | Hi all, I meet a bug when I set quota for a project. I am using kilo version of openstack. | 08:35 |
| wangqun | when I run the following command. | 08:35 |
| wangqun | openstack --debug quota set --volumes 20 623051a1ce4f417ea5b99e2bedca5e39 | 08:36 |
| wangqun | It will show the error. | 08:36 |
| wangqun | File "/usr/local/lib/python2.7/dist-packages/openstackclient/common/quota.py", line 124, in take_action | 08:36 |
| wangqun | **volume_kwargs) | 08:36 |
| wangqun | File "/usr/local/lib/python2.7/dist-packages/cinderclient/v1/quotas.py", line 47, in update | 08:36 |
| wangqun | result = self._update('/os-quota-sets/%s' % (tenant_id), body) | 08:36 |
| wangqun | I don't know how to fix the problem. | 08:37 |
| wangqun | Can anyone help me? | 08:37 |
| wangqun | ping alex_xu | 08:40 |
| *** jaosorior is now known as jaosorior_lunch | 08:40 | |
| *** moshele has quit IRC | 08:47 | |
| rpodolyaka | wangqun: that's an error in python-cinderclient and must have nothing to do with Nova | 08:48 |
| *** moshele has joined #openstack-nova | 08:48 | |
| rpodolyaka | unfortunately, it's only a part of the traceback, so it's hard to say what exactly went wrong | 08:48 |
| *** gongysh has joined #openstack-nova | 08:48 | |
| *** raj_singh_ has joined #openstack-nova | 08:49 | |
| mdbooth | edleafe: Thank you for change I6173274a03! I know that's from ages ago, but last time I used that log message it wasn't remotely that useful. | 08:50 |
| *** raj_singh_ has quit IRC | 08:53 | |
| *** derekh has joined #openstack-nova | 08:55 | |
| mdbooth | bauzas: I assume, btw, that DiskFilter not knowing that Rbd doesn't require local disk is a known issue. Under the ResourceProvider umbrella? | 08:56 |
| *** thorst has joined #openstack-nova | 08:56 | |
| *** pilgrimstack1 has quit IRC | 08:57 | |
| danpb | mdbooth: DiskFilter in general is utterly fubar with anything except 100% local non-shared disk | 08:58 |
| mdbooth | danpb: Was guessing as much :) Just checking. | 08:58 |
| danpb | iirc, recommendation is to not use DiskFilter | 08:59 |
| * mdbooth anticipated the recommendation \o/ | 08:59 | |
| *** code-R has joined #openstack-nova | 09:00 | |
| mdbooth | danpb: Incidentally, I've now managed to get a 2G instance with 2 vcpus which can live migration in my devstack environment | 09:00 |
| mdbooth | This is what I'm going to use to try to usefully force post-copy | 09:00 |
| *** thorst has quit IRC | 09:01 | |
| mdbooth | With smaller disks it just wasn't getting anywhere | 09:01 |
| *** pilgrimstack has joined #openstack-nova | 09:01 | |
| *** thorst has joined #openstack-nova | 09:01 | |
| mdbooth | If this doesn't work I may switch the nics to something a bit slower | 09:01 |
| mdbooth | danpb: Does realtek support 1G? | 09:02 |
| danpb | even with 1 Gig-E networking, if you use the stress program it should be possible to make migration fail with a mere 1 GB guest | 09:02 |
| *** code-R_ has joined #openstack-nova | 09:02 | |
| danpb | mdbooth: guest nic models have no bearing on migration success / failure | 09:03 |
| mdbooth | I mean the virtual nic of the vm which is running the compute host | 09:03 |
| *** maestropandy has joined #openstack-nova | 09:03 | |
| mdbooth | Assuming that if I switch those to something slower it'll all slow down | 09:03 |
| danpb | oh right you're using nested virt | 09:03 |
| mdbooth | Yeah | 09:03 |
| *** thorst has quit IRC | 09:04 | |
| *** jsheeren has joined #openstack-nova | 09:04 | |
| danpb | mdbooth: setup a traffic filter on your guest nic | 09:04 |
| danpb | http://libvirt.org/formatnetwork.html#elementQoS | 09:04 |
| danpb | the 'peak' attribute can be used to throttle it | 09:04 |
| mdbooth | danpb: Ooh, nice | 09:04 |
| *** code-R has quit IRC | 09:05 | |
| openstackgerrit | Zhenyu Zheng proposed openstack/nova: Should not be able to sort instances based on joined tables https://review.openstack.org/359134 | 09:09 |
| *** thorst has joined #openstack-nova | 09:09 | |
| *** lixiaoy1 has quit IRC | 09:12 | |
| *** thorst has quit IRC | 09:15 | |
| prateek_ | alex_xu, can you please look into https://review.openstack.org/#/c/327706/ | 09:15 |
| *** Jeffrey4l__ has joined #openstack-nova | 09:17 | |
| prateek_ | does anyone here has a link for setting up a devstack environ for live migration ? I am trying to set up using this http://docs.openstack.org/developer/devstack/guides/multinode-lab.html but am getting errors | 09:17 |
| ratailor | openstack, can any core-reviewer pl review https://review.openstack.org/#/c/312887/ | 09:18 |
| *** sridharg has joined #openstack-nova | 09:19 | |
| *** thorst has joined #openstack-nova | 09:19 | |
| *** thorst has quit IRC | 09:20 | |
| *** lucasagomes is now known as lucas-relocate | 09:21 | |
| *** sridhargaddam has quit IRC | 09:22 | |
| mdbooth | danpb: On a related note, my question yesterday about polling vs async notifications from libvirt | 09:25 |
| mdbooth | AFAICT the only thing nova is doing today is polling | 09:25 |
| mdbooth | Were you suggesting that there's a better way? | 09:25 |
| *** Zoya has joined #openstack-nova | 09:26 | |
| danpb | mdbooth: that's not true - nova listens for async events too | 09:26 |
| Zoya | Hi All, | 09:26 |
| Zoya | I have an issue | 09:26 |
| danpb | mdbooth: when it constructs the nova.virt.libvirt.host.Host object | 09:26 |
| mdbooth | danpb: Do you happen to know where? | 09:26 |
| * mdbooth looks there | 09:27 | |
| *** Zoya is now known as Guest60381 | 09:27 | |
| danpb | mdbooth: it passes in a callback that is invoked | 09:27 |
| danpb | the Host object has a native thread that gets events from libvirt and dispatches them into the greenthread | 09:27 |
| prateek_ | kashyap, can you please point me out to how you set up devstack for live migration ? | 09:27 |
| mdbooth | danpb: Ah, that gets stopped/started/paused/resumed | 09:27 |
| danpb | mdbooth: you'll need to extend the Host object though, as currently it only reports out the event type | 09:27 |
| danpb | mdbooth: and discards the event reason | 09:28 |
| mdbooth | So, the live migration monitor loop is polling for this info | 09:28 |
| danpb | s/reason/detail/ | 09:28 |
| mdbooth | Practically speaking, I'm going to have to stick with the polling approach for now, I think | 09:28 |
| danpb | and you'd need the detail to see that the pause/resume event was triggered by post-copy | 09:28 |
| mdbooth | Because mixing polling and async is a nightmare I don't want just for this | 09:28 |
| alex_xu | wangqun: you should file a bug and put all the error info into the bug report first | 09:28 |
| danpb | well if you're doing this inside the live migration monitor thread | 09:28 |
| mdbooth | danpb: But it sounds like the whole loop could be replaced with async? | 09:28 |
| danpb | then you don't need to poll either | 09:29 |
| alex_xu | prateek_: yea, will try to reach that tonight | 09:29 |
| kashyap | prateek_: Some configs and notes are here - https://kashyapc.fedorapeople.org/virt/openstack/multi-node-configs/ | 09:29 |
| danpb | because that thread knows exactly when it invokes the switch to post-copy | 09:29 |
| prateek_ | alex_xu, ok | 09:29 |
| Guest60381 | while booting a vm in Single server environment using flavor with extra_specs,getting this error: | 09:29 |
| prateek_ | kashyap, thanks | 09:29 |
| Guest60381 | {"message": "Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 2444e8c5-fa03-43bf-8941-b513e07e2b75. Last exception: XML error: Element 'memnode' is invalid without any guest NUMA cells", "code": 500, "details": " File \"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py\", line 393, in build_instances | 09:29 |
| *** PaulMurray has joined #openstack-nova | 09:29 | |
| Guest60381 | can anyone help me out | 09:29 |
| wangqun | alex_xu: Thanks. I will do it. | 09:29 |
| mdbooth | danpb: Right. But the loop itself polls. | 09:29 |
| mdbooth | It polls get_job_info() | 09:30 |
| mdbooth | I guess if all of the state changes it's looking for have associated events, we could replace the polling | 09:30 |
| danpb | no, afraid not, because the loop does much more stuff that's not related to state transitiions | 09:30 |
| danpb | eg it is dynamically changing max downtime throughout the migration process | 09:31 |
| mdbooth | Ok, cool. | 09:31 |
| danpb | as well as updating progress data on the migration object | 09:31 |
| mdbooth | I don't see any reason to do the networking stuff anywhere else, btw. | 09:31 |
| danpb | yeah, makes sense | 09:31 |
| *** praveen has joined #openstack-nova | 09:32 | |
| *** jsheeren has quit IRC | 09:33 | |
| *** thorst has joined #openstack-nova | 09:37 | |
| *** thorst has quit IRC | 09:38 | |
| *** thorst has joined #openstack-nova | 09:38 | |
| ratailor | johnthetubaguy, could you pl review https://review.openstack.org/#/c/312887/ | 09:38 |
| *** thorst has quit IRC | 09:39 | |
| *** thorst has joined #openstack-nova | 09:40 | |
| *** ducttape_ has joined #openstack-nova | 09:41 | |
| *** pilgrimstack has quit IRC | 09:42 | |
| *** diga has quit IRC | 09:43 | |
| *** mvk has quit IRC | 09:43 | |
| *** mfedosin has joined #openstack-nova | 09:45 | |
| *** pilgrimstack has joined #openstack-nova | 09:45 | |
| *** ducttape_ has quit IRC | 09:46 | |
| *** thorst has quit IRC | 09:51 | |
| Guest60381 | Hi All | 09:52 |
| Guest60381 | getting this error while nova booting: | 09:53 |
| Guest60381 | {"message": "Exceeded maximum number of retries. Exceeded max scheduling attempts 3 for instance 2444e8c5-fa03-43bf-8941-b513e07e2b75. Last exception: XML error: Element 'memnode' is invalid without any guest NUMA cells", "code": 500, "details": " File \"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py\", line 393, in build_instances | 09:53 |
| Guest60381 | using flavor with extra specs | 09:53 |
| Guest60381 | can anyone help me out | 09:53 |
| sfinucan | Guest60381: what version of nova? | 09:53 |
| openstackgerrit | Gleb Stepanov proposed openstack/nova: Pin HTTP response codes to API microversions https://review.openstack.org/305369 | 09:55 |
| Guest60381 | <sfinucan> Its 13.0.0 | 09:55 |
| Guest60381 | I think its problem with flavor | 09:56 |
| sfinucan | Guest60381: Can you paste and share a copy of your flavor ('openstack flavor show') here http://paste.openstack.org/ | 09:56 |
| Guest60381 | as I have included extra specs | 09:56 |
| sfinucan | Guest60381: yeah, but AFAICT nova should stop you doing something silly like that | 09:56 |
| sfinucan | :) | 09:56 |
| ratailor | johnthetubaguy, thanks :) | 09:57 |
| Guest60381 | <sfinucan> http://paste.openstack.org/show/569086/ | 09:57 |
| Guest60381 | <sfinucan>: Do I need to set anything related to nova | 09:58 |
| sfinucan | Guest60381: Thanks. Ok, drop the hugepages extra_spec first and see if that boots | 09:58 |
| Guest60381 | I have disabled this | 09:59 |
| sfinucan | Guest60381: no, if you're using upstream nova with the libvirt driver then it should just work (TM) | 09:59 |
| *** annp has quit IRC | 09:59 | |
| *** kevinz has joined #openstack-nova | 09:59 | |
| Guest60381 | ok..fine | 10:01 |
| sfinucan | Guest60381: so you're flavor should look exactly the same but without the 'hw:mem_page_size' parameter. Does that boot? | 10:01 |
| sfinucan | *your | 10:01 |
| Guest60381 | <sfinucan> Sorry I dn't know.. iam new to this concept :) | 10:04 |
| sfinucan | Guest60381: just create a new flavour like before but without that parameter :) | 10:04 |
| *** lucas-relocate is now known as lucasagomes | 10:05 | |
| Guest60381 | <sfinucan> without extra_specs parameters to flavor, nova boot was successful | 10:05 |
| *** GB21 has quit IRC | 10:05 | |
| sfinucan | Guest60381: OK. Now create a new flavor with all the extra specs except for 'hw:mem_page_size' and boot with that | 10:06 |
| *** liuyulong is now known as liuyulong|away | 10:06 | |
| *** gszasz has joined #openstack-nova | 10:06 | |
| Guest60381 | <sfinucan> http://paste.openstack.org/show/569139/ | 10:07 |
| Guest60381 | with this flavor also, got the same error | 10:07 |
| sfinucan | Guest60381: I don't know if that's valid. I don't recognise the 'hw:cpu_list' parameter | 10:08 |
| sfinucan | Guest60381: something like this http://paste.openstack.org/show/569140/ | 10:09 |
| *** hoangcx has quit IRC | 10:10 | |
| danpb | yeah, there is no such 'hw:cpu_list' property in nova - perhaps that's some out of tree downstream vendor addition | 10:10 |
| *** GB21 has joined #openstack-nova | 10:11 | |
| Guest60381 | <sfinucan> Without this "hw:cpu_list" parameter, able to boot the Vm | 10:12 |
| Guest60381 | :) | 10:13 |
| Guest60381 | Thanks a lot | 10:13 |
| *** mvk has joined #openstack-nova | 10:13 | |
| sfinucan | Guest60381: np. Out of curiosity, where did you find the 'hw:cpu_list' parameter documented? | 10:14 |
| Guest60381 | Didn't remember..but i got to know from my colleague | 10:15 |
| rpodolyaka | johnthetubaguy: thanks for review! could you please take a look at the follow up patch https://review.openstack.org/347948 ? | 10:18 |
| johnthetubaguy | rpodolyaka: yeah, sorry I got distracted, getting back to that in a sec | 10:19 |
| rpodolyaka | no worries! just want to make sure you don't miss it :P | 10:20 |
| *** sridhargaddam has joined #openstack-nova | 10:20 | |
| *** gongysh has quit IRC | 10:21 | |
| *** sridharg has quit IRC | 10:23 | |
| mdbooth | danpb: I think I must have fundamentally misunderstood the post-copy networking problem, because I can't reproduce it. | 10:26 |
| *** cdent has joined #openstack-nova | 10:26 | |
| *** ababich has joined #openstack-nova | 10:27 | |
| mdbooth | I'm assuming that neutron is not behaving as I expect, which presumably means I don't understand neutron :) (Not a surprise there) | 10:27 |
| danpb | mdbooth: what network setup as you using | 10:27 |
| danpb | i wouldn't be surprised if it didn't impact linux-bridge | 10:27 |
| mdbooth | I can now reliably force post-copy | 10:27 |
| mdbooth | But 2 guests can continue to ping each other throughout | 10:27 |
| mdbooth | Even when I deliberately hold-up the post job which should fix networking | 10:28 |
| danpb | mdbooth: how long is the guest in the post-copy phase for ? | 10:28 |
| mdbooth | danpb: Seconds | 10:28 |
| *** cdent has quit IRC | 10:28 | |
| danpb | ok, long enough to see a ping delay | 10:28 |
| mdbooth | Like, maybe 2 seconds | 10:28 |
| mdbooth | However, I stuck a break point in the post job | 10:28 |
| mdbooth | So networking is not being reconfigured on the dest for an arbitrarily long time | 10:29 |
| danpb | so what neutron plugin have you got ? | 10:29 |
| mdbooth | And during that time, networking continues to work | 10:29 |
| mdbooth | Already you're exceeding my neutron knowledge :) | 10:29 |
| danpb | ok, does 'brctl show' show a bridge device with your guest's nics in it | 10:29 |
| mdbooth | Yes | 10:30 |
| danpb | if so, you're using linuxbridge, if not, i guess oopenvswitch | 10:30 |
| mdbooth | Ok, so I'm guessing I need to run the other one | 10:30 |
| openstackgerrit | Paul Carlton proposed openstack/nova: Update instance group during live migration https://review.openstack.org/339588 | 10:30 |
| danpb | yeah, i get the feeling it might need openvswitch usage | 10:30 |
| *** phuongnh has quit IRC | 10:30 | |
| mdbooth | Ok. Is there a simple devstack switch to make it do that? | 10:30 |
| danpb | i had it working once | 10:31 |
| danpb | (with emphasis on the "once") | 10:31 |
| mdbooth | Hehe, ok | 10:31 |
| * mdbooth sharpens his yak shears | 10:31 | |
| Shashi | Hello, I created devsatck and configured backend sucessfully. Now when I am trying to create an Instance, I am gettinng error as "Instance failed network setup after 1 attempt" | 10:34 |
| Shashi | Can anyone help me out here? | 10:34 |
| mdbooth | danpb: Is 'ml2' the linux bridge thing? | 10:36 |
| danpb | i don't think so | 10:37 |
| danpb | but then i also don't entirely understand neutron terminologyu | 10:37 |
| mdbooth | neutron appears to be configured to use 'ml2' | 10:37 |
| kashyap | mdbooth: "The Modular Layer 2 (ML2) neutron plug-in is a framework allowing OpenStack Networking to simultaneously use the variety of layer 2 networking technologies found in complex real-world data centers." | 10:38 |
| kashyap | mdbooth: This page explains it nicely -- http://docs.openstack.org/mitaka/networking-guide/config-ml2-plug-in.html | 10:38 |
| mdbooth | kashyap: I just want to break it :) | 10:38 |
| kashyap | "Linux bridge" is one of the "type drivers" | 10:39 |
| mdbooth | kashyap: Ah, ok. | 10:39 |
| mdbooth | So I want ml2 with open vswitch by the sounds of it | 10:39 |
| kashyap | danpb: Yes | 10:40 |
| kashyap | Err, mdbooth, I mean | 10:40 |
| kashyap | That's also what the default DevStack setup with Neutron gives you | 10:40 |
| mdbooth | danpb: Whoa, just got a soft-lockup on the host with the guest running your stress | 10:41 |
| danpb | mdbooth: the real host, or the virtual host | 10:41 |
| openstackgerrit | Eli Qiao proposed openstack/nova: Libvirt: Correct PERF_EVENTS_CPU_FLAG_MAPPING https://review.openstack.org/365842 | 10:41 |
| mdbooth | the level 1 virt host | 10:41 |
| mdbooth | i.e. the vm which is hosting the guest | 10:41 |
| mdbooth | the vm which is running a vm containing the guest | 10:42 |
| danpb | could be that vm was not scheduled by the real host | 10:42 |
| *** tbachman has quit IRC | 10:42 | |
| *** andrey-mp has joined #openstack-nova | 10:42 | |
| mdbooth | For 54s? | 10:42 |
| *** ducttape_ has joined #openstack-nova | 10:42 | |
| mdbooth | Was stuck in khugepaged | 10:42 |
| *** andrey-mp has left #openstack-nova | 10:43 | |
| mdbooth | I guess something could have triggered a ton of io | 10:43 |
| *** wangqun has quit IRC | 10:46 | |
| *** ducttape_ has quit IRC | 10:47 | |
| *** amotoki has quit IRC | 10:48 | |
| kashyap | mdbooth: Are you using F24 Kernels? | 10:48 |
| mdbooth | kashyap: Yes | 10:49 |
| mdbooth | Well, physical is running RHEL7 | 10:49 |
| kashyap | mdbooth: In the past I've reported upstream soft lockups, and they turned out to be real bugs -- https://bugzilla.kernel.org/show_bug.cgi?id=69491 | 10:49 |
| openstack | bugzilla.kernel.org bug 69491 in kvm "Booting into a guest on Intel Haswell (bare-metal) throws soft lockups [qemu-system-x86:911]" [Normal,Resolved: code_fix] - Assigned to virtualization_kvm | 10:49 |
| mdbooth | Nested vms are both F24 | 10:49 |
| kashyap | Ah, baremetal is RHEL-7 | 10:49 |
| johnthetubaguy | rpodolyaka: quick question on that patch, why does it move to using request context, rather than admin context when listing ComputeNode objects? | 10:49 |
| mdbooth | kashyap: So, there's a file called ml2_conf.ini | 10:50 |
| mdbooth | which contains: mechanism_drivers = openvswitch,linuxbridge | 10:50 |
| kashyap | mdbooth: Yes, I was going to paste mine, the contents should look like this - http://paste.openstack.org/show/569150/ | 10:50 |
| kashyap | That's the output of `grep -v ^$ ml2_conf.ini | grep -v ^#` | 10:50 |
| kashyap | mdbooth: At the end you'd see an [ovs] section | 10:50 |
| *** edand has quit IRC | 10:50 | |
| mdbooth | That looks more complex than I need | 10:50 |
| mdbooth | Although... that looks pretty much like the default, no? | 10:51 |
| kashyap | mdbooth: Exactly, you don't have to muck with it. | 10:51 |
| mdbooth | Ok, so what I need is to kill all traces of linuxbridge | 10:51 |
| *** sudipto has quit IRC | 10:52 | |
| *** sudipto_ has quit IRC | 10:52 | |
| *** PaulMurray has quit IRC | 10:52 | |
| *** dikonoor has quit IRC | 10:53 | |
| kashyap | mdbooth: How did you end up with Linux Bridge (as opposed to OVS)? Did you explicitly configure it? | 10:56 |
| mdbooth | kashyap: No, I know nothing of neutron, so I touched nothing. Defaults all the way. | 10:56 |
| kashyap | Having this in your local.conf should directly bring you OVS -- ENABLED_SERVICES= [...] quantum,q-svc,q-agt,q-dhcp,q-l3,q-meta | 10:56 |
| *** baoli has joined #openstack-nova | 10:57 | |
| mdbooth | kashyap: I have that, and my setup is using neutron with ml2 | 10:58 |
| rpodolyaka | johnthetubaguy: iirc there was a comment in one of the previous patch sets that we do not really need an admin context, and elevated() is preferred | 10:58 |
| *** cdent has joined #openstack-nova | 10:58 | |
| mdbooth | The guests seem to be using linuxbridge, which seems to be something which ml2 also does | 10:58 |
| johnthetubaguy | rpodolyaka: sorry, found those comments now, I see what the other was updated now | 10:58 |
| mdbooth | I'm just trying again with a configuration which explicitly removes 'linuxbridge' from the ml2 config | 10:58 |
| kashyap | Ah, right. Hope simply restarting Neutron services would suffice | 10:59 |
| rpodolyaka | johnthetubaguy: no problem! | 10:59 |
| mdbooth | Heh, already blown everything away :) | 10:59 |
| *** rpodolyaka is now known as rpodolyakaLunch | 10:59 | |
| mdbooth | Just coming back up, ml2_conf.ini now has: | 11:00 |
| mdbooth | mechanism_drivers = openvswitch | 11:00 |
| mdbooth | no linuxbridge | 11:00 |
| *** bkopilov has quit IRC | 11:01 | |
| *** bkopilov_ has quit IRC | 11:02 | |
| *** baoli has quit IRC | 11:05 | |
| *** sdague has joined #openstack-nova | 11:06 | |
| *** jsheeren has joined #openstack-nova | 11:08 | |
| mdbooth | paul-carlton2: Do you know anything about https://bugs.launchpad.net/nova/+bug/1605016 | 11:09 |
| openstack | Launchpad bug 1605016 in OpenStack Compute (nova) "Post copy live migration interrupts network connectivity" [High,Confirmed] - Assigned to Matthew Booth (mbooth-9) | 11:09 |
| mdbooth | Specifically, how to reproduce it? | 11:09 |
| jsheeren | hi all | 11:09 |
| paul-carlton2 | a bit | 11:09 |
| mdbooth | I'm having trouble creating a neutron configuration which fails in the manner described. | 11:09 |
| jsheeren | when we launch an instane, nova updates the mtime for the _base image to let the image cache manager know the image is activaly used | 11:10 |
| paul-carlton2 | mdbooth, I think the issue is that if the post copy phase take a while you will see a network outage | 11:10 |
| jsheeren | i'm seeing errors in my nova compute log, where it cannot do this. triggering a reschedule on another compute node | 11:10 |
| mdbooth | paul-carlton2: That's the issue. I don't see an outage. | 11:10 |
| mdbooth | jsheeren: Why would it not be able to do that? | 11:11 |
| jsheeren | for my setup i have 3 compute nodes. i'm seeing this error on all three compute nodes | 11:11 |
| mdbooth | jsheeren: That's not an expected failure. | 11:11 |
| paul-carlton2 | mdbooth, suspect in most test scenarios the post copy phase lasts a second or so | 11:11 |
| jsheeren | well, that 's why I'm here :-) | 11:11 |
| jsheeren | we're using an nfs export for shared storage | 11:11 |
| mdbooth | jsheeren: What is the failure when it tries to update the mtime? | 11:11 |
| mdbooth | paul-carlton2: Nah, I've artificially extended it. | 11:11 |
| mdbooth | No network outage at all. | 11:11 |
| *** trisq has quit IRC | 11:12 | |
| jsheeren | mbooth: Stderr: u"/bin/touch: setting times of '/var/lib/nova/instances/_base/79e34519bacb47ad6f64c4baca4d33fd5c57d34d': Input/output error | 11:12 |
| jsheeren | this error particulary shows itself when launching multiple instances at once | 11:12 |
| mdbooth | jsheeren: Can you normally touch one of those files? | 11:12 |
| jsheeren | yes | 11:12 |
| jsheeren | as nova user with rootwrap | 11:12 |
| mdbooth | At first touch I'd suspect a storage issue | 11:12 |
| jsheeren | and offcourse as root user | 11:12 |
| jsheeren | we're using an nfs4.1 export | 11:13 |
| paul-carlton2 | mdbooth, you sure it is not talking to source instance? | 11:13 |
| *** baoli has joined #openstack-nova | 11:13 | |
| mdbooth | paul-carlton2: Well the source instance shouldn't still be running, right? | 11:13 |
| mdbooth | That would be bad. | 11:13 |
| jsheeren | can it be that when launching say 20 instnaces at once, we're seeing some sort of race conditions? | 11:13 |
| mdbooth | jsheeren: For sure, but I wouldn't expect a race to manifest like that. | 11:14 |
| *** kevinz has quit IRC | 11:14 | |
| paul-carlton2 | mdbooth, it should pause it when it switches to post copy I'd have thought but worth checking | 11:14 |
| mdbooth | For a race failure I'd expect something like trying to update the mtime on a file which doesn't exist. | 11:14 |
| mdbooth | However io/error trying to touch a file which does exist... | 11:14 |
| mdbooth | That just smells like storage config to me | 11:15 |
| jsheeren | mbooth: using default nfs options, like local_lock=none. so i would assume that would be safe enough | 11:15 |
| jsheeren | can it be we're experiencing some sort of latencies then? (i'm not an nfs expert .heh) | 11:15 |
| mdbooth | jsheeren: The question is why you get an io error when running touch | 11:16 |
| paul-carlton2 | mdbooth, if you are connected to an instance via ssh and do a migration with extended post copy phase I'd expect the ssh session to freeze when past copy switch occurs, but if you check the source instance and find it is not paused then that would explain it | 11:16 |
| prateek_ | johnthetubaguy, can you please review ? | 11:16 |
| mdbooth | EPERM Or EEXIST would obviously be nova | 11:16 |
| prateek_ | johnthetubaguy, can you please review https://review.openstack.org/#/c/294513/ ? | 11:17 |
| mdbooth | paul-carlton2: The issue as I understood it before I tried to reproduce | 11:17 |
| mdbooth | Is that networking isn't reconfigured on the destination until the migration completes | 11:17 |
| paul-carlton2 | yep | 11:18 |
| mdbooth | So, I stuck a breakpoint there | 11:18 |
| mdbooth | Then I forced post-copy migration | 11:18 |
| mdbooth | And hit my breakpoint | 11:18 |
| paul-carlton2 | so at post copy you should see source paused, dest running, is that the case? | 11:18 |
| jsheeren | on a side note; because of the reschedules on other compute nodes, we get multiple private ips for the instances; which is annoying | 11:19 |
| paul-carlton2 | i.e. once post copy switch is complete | 11:19 |
| mdbooth | Well, when you hit the breakpoint the migration is complete | 11:19 |
| mdbooth | But networking has not yet been reconfigured | 11:19 |
| *** maestropandy has quit IRC | 11:19 | |
| mdbooth | For non-post-copy this happens straight away | 11:19 |
| jsheeren | this is similar to bug https://bugs.launchpad.net/nova/+bug/1609526 | 11:19 |
| openstack | Launchpad bug 1609526 in OpenStack Compute (nova) "nova should clean up network ports when an image fails to download from glance" [Undecided,Confirmed] - Assigned to Zhenyu Zheng (zhengzhenyu) | 11:19 |
| mdbooth | For post-copy the delay could be longer | 11:19 |
| *** baoli has quit IRC | 11:19 | |
| mdbooth | However, in either case, networking should work yet, right? | 11:19 |
| mdbooth | s/should/shouldn't/ | 11:20 |
| paul-carlton2 | mdbooth, you know the libvirt/qemu code better than me but surely you need the breakpoint after switch to target instance but before all remaining pages of memory are copied | 11:20 |
| openstackgerrit | jichenjc proposed openstack/nova: no instance info cache update if instance deleted https://review.openstack.org/363585 | 11:20 |
| *** coolsvap is now known as _coolsvap_ | 11:21 | |
| mdbooth | paul-carlton2: Unless I'm still misunderstanding (possible!) libvirt/qemu is irrelevant here | 11:21 |
| mdbooth | Because networking shouldn't work until post_live_migration completes | 11:21 |
| mdbooth | That should be true for post-copy or non-post-copy | 11:21 |
| *** sridharg has joined #openstack-nova | 11:22 | |
| mdbooth | The additional complication for post-copy is that we do the switch before the migration completes | 11:22 |
| mdbooth | So in the intervening period we have no networking | 11:22 |
| mdbooth | Except that we do | 11:22 |
| paul-carlton2 | mdbooth, as I understand it the issue is that at present nova fires of libvirt/qemu migration and waits for it complete then when it is done switches networking | 11:22 |
| openstackgerrit | jichenjc proposed openstack/nova: prevent non-admin user search soft-delete server https://review.openstack.org/258472 | 11:22 |
| *** salv-orl_ has quit IRC | 11:22 | |
| mdbooth | Right, so I stuck the breakpoint before the code which switches the networking | 11:22 |
| mdbooth | So I've got a migrated guest whose networking has not yet been switched | 11:23 |
| *** dikonoor has joined #openstack-nova | 11:23 | |
| mdbooth | But its networking still works anyway | 11:23 |
| *** sridhargaddam has quit IRC | 11:24 | |
| *** baoli has joined #openstack-nova | 11:24 | |
| paul-carlton2 | mdbooth, if post copy is used then at some point during processing of live migration by qemu it is told by nova to switch to post copy mode at which poit it should pause the source instance and resume the target instance so target is now active instance and starts doing work | 11:24 |
| paul-carlton2 | however at this point network is still connected to source instance | 11:24 |
| *** roeyc has quit IRC | 11:25 | |
| mdbooth | jsheeren: I believe there are lots of races in that code, btw. What you're describing may be the symptom of a bug in Nova, but unless we can work out why an NFS operation returns EIO, we don't have much chance of working out what's going on. | 11:25 |
| mdbooth | And it's also possible it's not a Nova bug at all, just a storage configuration error. | 11:25 |
| mdbooth | paul-carlton2: Right, but it works anyway. | 11:25 |
| paul-carlton2 | mdbooth, if source instance is paused then the open connections will hang and new attemptes will fail, is that not what you are seeing? | 11:26 |
| mdbooth | paul-carlton2: So, iiuc this has nothing to do with the source instance. | 11:27 |
| mdbooth | And in fact, post-copy is an irrelevance, as at this point in the code the migration has completed | 11:27 |
| mdbooth | So, lets ignore post-copy for a moment. | 11:27 |
| mdbooth | The migration has completed | 11:27 |
| *** rtheis has joined #openstack-nova | 11:27 | |
| mdbooth | At this point, the guest is running on the destination, but its networking has not been reconfigured yet | 11:27 |
| mdbooth | My expectation was that guest networking would not work | 11:28 |
| mdbooth | However, it does | 11:28 |
| *** dikonoor has quit IRC | 11:28 | |
| *** dikonoor has joined #openstack-nova | 11:29 | |
| paul-carlton2 | mdbooth, yes if you let libvirt/qemu complete migration but prevent nova from switching network then networking is still connected to source instance | 11:29 |
| danpb | paul-carlton2: that's not in itself a problem | 11:29 |
| mdbooth | paul-carlton2: Yes, that's exactly what's so confusing | 11:29 |
| mdbooth | Because despite that, it still works | 11:29 |
| danpb | IIUC it is only an issue for network topologies where neutron must manually switch a port to being active | 11:29 |
| mdbooth | So, has neutron created us a bridge which it has previously spanned across both hosts? | 11:30 |
| danpb | linux bridge for example "just works" with no manual steps required as bridges just learn the new topology when qemu sends the gratuitous ARP after migration | 11:30 |
| paul-carlton2 | mdbooth, I had imagined that when libvirt sees migration by qemu complete it pauses (or destroys) source and unpauses target, no? | 11:30 |
| *** baoli has quit IRC | 11:30 | |
| mdbooth | So when the migrated guest connects to it, it can already arp | 11:30 |
| danpb | i think its only an issue for network systems can't can't automatically re-configure themsellves based on he ARP | 11:30 |
| mdbooth | paul-carlton2: So, I think what danpb is describing is that this issue is completely irrelevant if you're using bridging. | 11:32 |
| mdbooth | I was wondering if anybody in your team had a reproducer | 11:33 |
| mdbooth | Which would presumably involve a different neutron setup. | 11:33 |
| mdbooth | I'm hoping to shortcut working out the neutron config | 11:33 |
| *** fragatina has joined #openstack-nova | 11:36 | |
| paul-carlton2 | mdbooth, not sure anyone here has worked on this, you'd need a neutron networking setup such that the source and destination compute hosts require different routing to access instance | 11:37 |
| *** gongysh has joined #openstack-nova | 11:37 | |
| *** zenoway has quit IRC | 11:37 | |
| paul-carlton2 | mdbooth, afraid I don't know enough about networking to know how to create such a scenario, could you just supress arp'ing? | 11:38 |
| mdbooth | I think I'm going to have to go ask the neutron folks | 11:38 |
| paul-carlton2 | mdbooth, good plan, by the way it seems arp suppression is easy http://linux-ip.net/html/ether-arp.html | 11:40 |
| *** dikonoor has quit IRC | 11:40 | |
| *** fragatina has quit IRC | 11:41 | |
| *** panbalag has joined #openstack-nova | 11:43 | |
| *** baoli has joined #openstack-nova | 11:43 | |
| *** ducttape_ has joined #openstack-nova | 11:43 | |
| openstackgerrit | Alex Xu proposed openstack/nova: Ensure ResourceProvider/Inventory created before add Allocations record https://review.openstack.org/367307 | 11:44 |
| *** rfolco has joined #openstack-nova | 11:44 | |
| *** ducttape_ has quit IRC | 11:48 | |
| *** lucasagomes is now known as lucas-hungry | 11:48 | |
| *** tonytan4ever has joined #openstack-nova | 11:51 | |
| *** jsheeren has quit IRC | 11:51 | |
| *** links has quit IRC | 11:52 | |
| *** jaosorior_lunch is now known as jaosorior | 11:56 | |
| *** tonytan4ever has quit IRC | 11:56 | |
| *** baoli_ has joined #openstack-nova | 11:56 | |
| *** baoli has quit IRC | 12:00 | |
| *** rpodolyakaLunch is now known as rpodolyaka | 12:02 | |
| openstackgerrit | Ludovic Beliveau proposed openstack/nova: PCI: Fix PCI with fully qualified address https://review.openstack.org/363884 | 12:05 |
| *** jsheeren has joined #openstack-nova | 12:09 | |
| *** kylek3h_ has joined #openstack-nova | 12:09 | |
| *** mriedem has joined #openstack-nova | 12:10 | |
| *** ducttape_ has joined #openstack-nova | 12:11 | |
| *** kylek3h has quit IRC | 12:12 | |
| *** takedakn has joined #openstack-nova | 12:13 | |
| *** kylek3h_ has quit IRC | 12:13 | |
| mriedem | tonyb: claudiub|2: mikal: can i get you to take a look at this mitaka backport that's holding up another backport that sits on top of it? https://review.openstack.org/#/c/357566/ | 12:16 |
| mriedem | BobBall: is something up with the citrix xenserver ci? it's failing a lot recently it seems | 12:16 |
| mriedem | BobBall: i guess i'm seeing different issues in each failure run i'm seeing, so maybe it's just random | 12:18 |
| mriedem | wznoinsk: is the intel nfv ci supposed to be running on stable/liberty | 12:18 |
| mriedem | ? | 12:18 |
| *** thomasem has quit IRC | 12:18 | |
| wznoinsk | mriedem: it's disabled at the moment as it was giving rubbish results, I still need to look into it | 12:19 |
| mriedem | wznoinsk: ok it ran on https://review.openstack.org/#/c/366996/ yesterday and failed | 12:19 |
| wznoinsk | checking | 12:20 |
| *** wuhg has quit IRC | 12:21 | |
| *** thomasem has joined #openstack-nova | 12:22 | |
| *** sridhargaddam has joined #openstack-nova | 12:22 | |
| *** pradk has joined #openstack-nova | 12:22 | |
| BobBall | mriedem: http://ci-watch.tintri.com/project?project=nova doens't seem to show a consistent issue? | 12:24 |
| BobBall | mriedem: Got a linky I can look at? | 12:24 |
| wznoinsk | mriedem: I see a misconfig there, I'll ping you when it's fixed | 12:24 |
| *** gcb has quit IRC | 12:24 | |
| BobBall | mriedem: Or is this just on stable branches? | 12:25 |
| BobBall | mriedem: We know that we broke stable branches a few days ago - but fixed it today after Tony highlighted the issue (we were relying on some changes that haven't been backported) | 12:25 |
| *** praveen has quit IRC | 12:25 | |
| *** sridharg has quit IRC | 12:26 | |
| *** xinliang has quit IRC | 12:26 | |
| *** bkopilov_ has joined #openstack-nova | 12:27 | |
| *** kevinz has joined #openstack-nova | 12:28 | |
| *** toure has joined #openstack-nova | 12:28 | |
| mriedem | BobBall: https://review.openstack.org/#/c/366994/ | 12:29 |
| *** bkopilov has joined #openstack-nova | 12:29 | |
| mriedem | there was a stable/liberty devstack failure yesterday, that looked like a mirror issue | 12:29 |
| mriedem | wznoinsk: thanks | 12:30 |
| BobBall | OK - that latest failure is the one I mentioned we fixed this morning after Tony highlighted it | 12:30 |
| BobBall | It would have failed all stable branches I believe | 12:31 |
| BobBall | The devstack failure yesterday was fixed by a recheck; just a transient issue I guess | 12:31 |
| *** mdrabe has joined #openstack-nova | 12:31 | |
| *** tbachman has joined #openstack-nova | 12:31 | |
| *** EinstCrazy has joined #openstack-nova | 12:31 | |
| BobBall | Correction - the devstack failure was actually 3 weeks ago - hence the recheck to get a relatively recent attempt | 12:32 |
| *** akshai has joined #openstack-nova | 12:32 | |
| BobBall | See https://review.openstack.org/#/c/358152/ for the pass | 12:32 |
| *** lpetrut has quit IRC | 12:33 | |
| BobBall | So I'm going to say that I think everything is OK ATM? :) | 12:33 |
| johnthetubaguy | mriedem: I can take a peak at that backport for you | 12:34 |
| mriedem | BobBall: since i'm not seeing a pattern of the same failures i guess they are just random yeah | 12:35 |
| prateek_ | alex_xu, or anyone willing to answer :), one question suppose I have an operation throwing 500 and the correct error code is 409, which needs a microversion, so can i map the error to 400 error code and handle that ? | 12:35 |
| *** kevinz has quit IRC | 12:35 | |
| *** kevinz has joined #openstack-nova | 12:35 | |
| *** akshai_ has joined #openstack-nova | 12:36 | |
| *** su_zhang has joined #openstack-nova | 12:36 | |
| mriedem | johnthetubaguy: thanks | 12:36 |
| *** ducttape_ has quit IRC | 12:36 | |
| BobBall | mriedem: Do let me know if you see something though - we are enabling more tests ATM and while we haven't seen any increase in races (other than the stable branches, which we fixed today), it's quite possible that there are some. | 12:36 |
| *** GB21 has quit IRC | 12:36 | |
| *** lucas-hungry is now known as lucasagomes | 12:38 | |
| johnthetubaguy | mriedem: got all moved house now, so I am back around, let me know if there are urgent things, currently going through patches with +2s already, seeing which ones are ready to go | 12:38 |
| alex_xu | prateek_: the change from 500 to 409 needn't microversion | 12:38 |
| *** akshai has quit IRC | 12:39 | |
| alex_xu | mriedem: morning, I have one patch for back-port https://review.openstack.org/364613 | 12:39 |
| *** xinliang has joined #openstack-nova | 12:40 | |
| *** tbachman has quit IRC | 12:41 | |
| *** cdelatte has joined #openstack-nova | 12:42 | |
| mriedem | alex_xu: ok, we could almost just ignore that since i doubt anyone is using it, given we didn't even know it existed until 2.36 | 12:42 |
| *** kylek3h has joined #openstack-nova | 12:42 | |
| openstackgerrit | Ludovic Beliveau proposed openstack/nova: Don't parse PCI whitelist eveytime neutron ports are created https://review.openstack.org/337236 | 12:43 |
| bauzas | mriedem: do you know the OSC project name in Launchpad ? I need to punt https://bugs.launchpad.net/nova/+bug/1621391 to those folks | 12:45 |
| openstack | Launchpad bug 1621391 in OpenStack Compute (nova) "Error "Could not find resource cirros" even image is created" [Undecided,New] | 12:45 |
| prateek_ | alex_xu, https://github.com/openstack/nova/blob/master/doc/source/api_microversion_dev.rst says "When fixing 500 errors that previously caused stack traces, try to map the new error into the existing set of errors that API call could previously return (400 if nothing else is appropriate). Changing the set of allowed status codes from a request is changing the contract, and should be part of a microversion (except in [2]).," So that | 12:45 |
| prateek_ | means that a patch does not adhere to this, is it ? | 12:45 |
| mriedem | bauzas: https://bugs.launchpad.net/python-openstackclient | 12:46 |
| *** maestropandy has joined #openstack-nova | 12:46 | |
| *** burgerk has joined #openstack-nova | 12:46 | |
| mriedem | alex_xu: +W | 12:46 |
| * bauzas owes some stuff for mriedem | 12:47 | |
| *** tbachman has joined #openstack-nova | 12:47 | |
| *** amotoki has joined #openstack-nova | 12:48 | |
| mriedem | i'll accept payment in cheese form | 12:48 |
| *** diana_clarke has joined #openstack-nova | 12:49 | |
| *** mmedvede has quit IRC | 12:49 | |
| bauzas | mriedem: you're fortunate that Barcelona is in the Schengen area | 12:49 |
| *** jaosorior has quit IRC | 12:51 | |
| *** jaosorior has joined #openstack-nova | 12:51 | |
| *** vladikr has joined #openstack-nova | 12:53 | |
| bauzas | does someone know a bit of noVNC ? | 12:53 |
| bauzas | https://bugs.launchpad.net/nova/+bug/1621257 claims there is a regression but I'd bet it's unrelated to Nova, rather to the novnc package | 12:53 |
| openstack | Launchpad bug 1621257 in OpenStack Compute (nova) "VNC console keeps reporting "setkeycodes 00" exception" [Undecided,New] | 12:53 |
| *** dikonoor has joined #openstack-nova | 12:54 | |
| mriedem | bauzas: the only person i know of was solly ross | 12:54 |
| mriedem | who no longer works on it | 12:54 |
| *** maestropandy has quit IRC | 12:54 | |
| bauzas | yeah... | 12:55 |
| * bauzas looking at the last commits | 12:55 | |
| *** ratailor has quit IRC | 12:55 | |
| *** cleong has joined #openstack-nova | 12:56 | |
| openstackgerrit | Sean Dague proposed openstack/nova: Additional logging for placement API https://review.openstack.org/366186 | 12:56 |
| *** xyang1 has joined #openstack-nova | 12:57 | |
| *** liverpooler has quit IRC | 12:58 | |
| *** migi has quit IRC | 13:00 | |
| openstackgerrit | Sean Dague proposed openstack/nova: Additional logging for placement API https://review.openstack.org/366186 | 13:00 |
| sdague | mriedem: ok, that logging patch should be ready to go | 13:00 |
| *** maestropandy has joined #openstack-nova | 13:00 | |
| *** liverpooler has joined #openstack-nova | 13:01 | |
| *** tbachman has quit IRC | 13:02 | |
| *** pwnall has joined #openstack-nova | 13:03 | |
| *** pwnall is now known as Guest19167 | 13:03 | |
| *** oanson has quit IRC | 13:04 | |
| alex_xu | mriedem: thanks | 13:04 |
| alex_xu | prateek_: oops, you are right, I missed that point | 13:04 |
| *** prateek_ has quit IRC | 13:05 | |
| *** mingyu has joined #openstack-nova | 13:06 | |
| vladikr | mriedem, Hi, wanted to ask about https://review.openstack.org/#/c/335676/ if you have a moment. seems to me that I'll do what you are asking for it won't be testable.. | 13:06 |
| *** tonytan4ever has joined #openstack-nova | 13:07 | |
| alex_xu | sdague: do we still follow this rule http://docs.openstack.org/developer/nova/api_microversion_dev.html#f1 ? | 13:07 |
| *** tuanla_ has quit IRC | 13:07 | |
| *** raj_singh_ has joined #openstack-nova | 13:08 | |
| *** jheroux has joined #openstack-nova | 13:09 | |
| *** yamahata has joined #openstack-nova | 13:09 | |
| *** mingyu has quit IRC | 13:09 | |
| openstackgerrit | jichenjc proposed openstack/nova: Catch DevicePathInUse in attach_volume https://review.openstack.org/367358 | 13:10 |
| mriedem | vladikr: how is it that the code is not testable? | 13:10 |
| mriedem | w/o the refactor? | 13:11 |
| *** tblakes has joined #openstack-nova | 13:11 | |
| mriedem | vladikr: i don't really care if the refactor is before or after the bug fix, but it shouldn't be in the same change as the bug fix | 13:11 |
| mriedem | sdague: ok. breakfasting and then have to send my daughter off to the first day of kindergarten and try not to weep in public. | 13:12 |
| *** jraju has joined #openstack-nova | 13:12 | |
| *** tonytan4ever has quit IRC | 13:12 | |
| mriedem | but it's in a tab | 13:12 |
| vladikr | mriedem, yea, w/o sprinting that part out it will be very hard to test it. I'll split and make the fix on top. thanks | 13:13 |
| *** raj_singh_ has quit IRC | 13:13 | |
| vladikr | s/sprinting/splitting | 13:13 |
| *** mingyu has joined #openstack-nova | 13:13 | |
| sdague | alex_xu: yeh, is there a patch up somewhere that you are questioning? | 13:13 |
| mriedem | vladikr: cool | 13:13 |
| *** timello has quit IRC | 13:13 | |
| *** tonytan4ever has joined #openstack-nova | 13:14 | |
| alex_xu | sdague: thanks for confirm. no patch, just prateek ask me about that. | 13:15 |
| *** jsheeren has quit IRC | 13:15 | |
| *** PaulMurray has joined #openstack-nova | 13:15 | |
| *** eharney has joined #openstack-nova | 13:15 | |
| *** su_zhang has quit IRC | 13:16 | |
| *** su_zhang has joined #openstack-nova | 13:16 | |
| *** mmedvede has joined #openstack-nova | 13:17 | |
| *** jsheeren has joined #openstack-nova | 13:17 | |
| *** su_zhang_ has joined #openstack-nova | 13:20 | |
| *** sdake_ has joined #openstack-nova | 13:20 | |
| *** timello has joined #openstack-nova | 13:20 | |
| *** su_zhang has quit IRC | 13:20 | |
| dikonoor | mriedem: Hi | 13:21 |
| *** sdake has quit IRC | 13:22 | |
| *** jsheeren has quit IRC | 13:22 | |
| dikonoor | mriedem: I have a query about VM deploys using key-pair. | 13:22 |
| *** jsheeren has joined #openstack-nova | 13:23 | |
| *** sridharg has joined #openstack-nova | 13:23 | |
| *** aloga_ has joined #openstack-nova | 13:24 | |
| dikonoor | mriedem: I am not sure who's the right person I should be talking to ..if you can let me know great..The VM deploy flow currently uses the user_id from the context object to fetch the key_name | 13:24 |
| dikonoor | mriedem: and if there isn't a key with that name associated with that user, the deploy fails. | 13:24 |
| *** yamahata has quit IRC | 13:25 | |
| dikonoor | mriedem: We have a specific case where we want the admin to be able to deploy VM for a user such that the key that corresponds to a user (other than the admin himself) is used. | 13:25 |
| *** sridhargaddam has quit IRC | 13:26 | |
| dikonoor | mriedem: and that flow obviously does not work here.. An admin user can create a key-pair for a specific user but not use that key-pair to do deploys.. | 13:27 |
| *** ducttape_ has joined #openstack-nova | 13:28 | |
| *** liverpooler has quit IRC | 13:28 | |
| *** baoli_ has quit IRC | 13:29 | |
| openstackgerrit | Sam Betts proposed openstack/nova: [WIP] Support Ironic interface attach/detach in nova virt https://review.openstack.org/364413 | 13:29 |
| *** jsheeren_ has joined #openstack-nova | 13:30 | |
| *** liverpooler has joined #openstack-nova | 13:30 | |
| *** takedakn has quit IRC | 13:31 | |
| dikonoor | alex_xu: perhaps mriedem is not in yet.. Would you know whom I should be talking to on this ? | 13:32 |
| *** mingyu has quit IRC | 13:32 | |
| *** jsheeren has quit IRC | 13:33 | |
| johnthetubaguy | I think he has just headed out but he is back in a bit | 13:33 |
| *** xenogear has quit IRC | 13:34 | |
| *** jinli has quit IRC | 13:34 | |
| *** MikeG451 has quit IRC | 13:34 | |
| *** stewie925 has quit IRC | 13:34 | |
| *** cmartin has quit IRC | 13:34 | |
| *** syjulian has quit IRC | 13:34 | |
| *** salv-orlando has joined #openstack-nova | 13:34 | |
| dikonoor | johnthetubaguy: Thanks..Would you be able to take my query ? | 13:34 |
| johnthetubaguy | dikonoor: AFAIK, the admin can't create a vm on the users behalf at the moment, unless they can get an impersonation token | 13:34 |
| johnthetubaguy | the key pair stuff was more about listing and deleting them, I can't remember the specific use case now | 13:35 |
| *** jsheeren_ has quit IRC | 13:35 | |
| *** stewie925 has joined #openstack-nova | 13:36 | |
| dikonoor | johnthetubaguy: yes..admin cannot because the key_name is mapped to another user.. the other way to get this working would be to pass a user_id attribute as part of the request body and use that to fetch the key-pair . If the user_id isn't present in the request, then use the user_id in the context | 13:37 |
| johnthetubaguy | dikonoor: right, its an API feature we don't support today | 13:37 |
| dikonoor | johnthetubaguy: this is similar to how key-pairs are created. | 13:37 |
| cdent | sdague: there's a potential wart in your logging patch, I offered up some ways out depending on your goal | 13:37 |
| *** mingyu has joined #openstack-nova | 13:38 | |
| *** ttrumm has quit IRC | 13:38 | |
| dikonoor | johnthetubaguy: the create key-pair flow first checks if there's a userid in the request body .If not present, use it from the context | 13:38 |
| *** gongysh has quit IRC | 13:38 | |
| *** ttrumm has joined #openstack-nova | 13:38 | |
| dikonoor | http://developer.openstack.org/api-ref-compute-v2.1.html | 13:39 |
| *** jamesdenton has joined #openstack-nova | 13:40 | |
| sdague | cdent: ah, yeh, the get_all_by_consumer_id sounds right | 13:40 |
| sdague | cdent: let me respin that | 13:40 |
| cdent | sdague: sorry my comment is not quite correct | 13:40 |
| *** ducttape_ has quit IRC | 13:40 | |
| dikonoor | johnthetubaguy: if we need to add support to consider the user_id in the request body so that admins can use the keys created by the users in a particular project , then would that be a nova defect or does it need like a blueprint? | 13:40 |
| cdent | you can do the following three: | 13:41 |
| johnthetubaguy | dikonoor: that needs to be a spec, its a new API feature | 13:41 |
| *** ducttape_ has joined #openstack-nova | 13:41 | |
| dikonoor | johnthetubaguy : I mean for the deploy flow | 13:41 |
| *** ducttape_ has quit IRC | 13:41 | |
| cdent | AllocationList.get_all_by{consumer_id,resource_provider_uuid}, UsageList.get_all_by_resource.. | 13:41 |
| dikonoor | johnthetubaguy: ok..and If I am not wrong, specs are closed for newton and it has to be proposed for ocata | 13:41 |
| cdent | you _can't_ do UsageList.get_all_by_consumer_id | 13:41 |
| *** ducttape_ has joined #openstack-nova | 13:41 | |
| sdague | cdent: yeh, right, that just sunk in | 13:41 |
| johnthetubaguy | dikonoor: I am not sure if keystone should be in charge of that kind of thing though, but thats a deeper discussion for the spec | 13:42 |
| johnthetubaguy | dikonoor: thats correct, proposed of ocata is the option now | 13:42 |
| *** salv-orlando has quit IRC | 13:42 | |
| *** ttrumm has quit IRC | 13:43 | |
| dikonoor | johnthetubaguy: ok ..sure..Thanks .. Will work on proposing a spec for this then.. | 13:43 |
| johnthetubaguy | dikonoor: cool, we can think through the details in there | 13:43 |
| dikonoor | johnthetubaguy: sure | 13:44 |
| *** hshiina has joined #openstack-nova | 13:44 | |
| johnthetubaguy | dikonoor: just curious, is this related to people who are admins creating instances, or other services creating instances? | 13:44 |
| *** _ducttape_ has joined #openstack-nova | 13:44 | |
| *** _ducttape_ has quit IRC | 13:45 | |
| *** woodster_ has joined #openstack-nova | 13:45 | |
| dikonoor | johnthetubaguy: its the former... admins creating instance on behalf on other users within the same project | 13:45 |
| *** _ducttape_ has joined #openstack-nova | 13:45 | |
| *** rlrossit has joined #openstack-nova | 13:47 | |
| *** roeyc has joined #openstack-nova | 13:47 | |
| *** ducttape_ has quit IRC | 13:48 | |
| *** jinli has joined #openstack-nova | 13:48 | |
| johnthetubaguy | dikonoor: OK, thanks, good to know | 13:48 |
| *** xenogear has joined #openstack-nova | 13:50 | |
| *** brian_price has quit IRC | 13:50 | |
| *** bhagyashri has quit IRC | 13:51 | |
| dansmith | sdague: are you reviewing the GET on allocations patch? | 13:53 |
| sdague | dansmith: I am not, which patch do you want me to look at? | 13:53 |
| dansmith | oh I thought maybe it was the reason for the above discussion | 13:54 |
| dansmith | https://review.openstack.org/#/c/366789 and the patch that depends on it | 13:54 |
| mdbooth | dansmith: https://bugs.launchpad.net/nova/+bug/1605016 | 13:54 |
| openstack | Launchpad bug 1605016 in OpenStack Compute (nova) "Post copy live migration interrupts network connectivity" [High,Confirmed] - Assigned to Matthew Booth (mbooth-9) | 13:54 |
| mdbooth | Or mriedem ^^^ | 13:54 |
| sdague | dansmith: no, that was the logging one | 13:54 |
| mdbooth | I'm looking for assistence with devstack | 13:54 |
| sdague | cdent: I updated the logging one | 13:54 |
| *** liverpooler has quit IRC | 13:55 | |
| dansmith | sdague: okay | 13:55 |
| *** takashin has joined #openstack-nova | 13:56 | |
| *** brian_price has joined #openstack-nova | 13:56 | |
| sdague | cdent: why aren't allocations just a list here - https://review.openstack.org/#/c/366789/1 ? | 13:56 |
| mriedem | mdbooth: devstack is neutron by default now | 13:56 |
| *** jaypipes has joined #openstack-nova | 13:56 | |
| *** roeyc has quit IRC | 13:57 | |
| mriedem | mdbooth: are you looking for devstack multinode setup to recreate? | 13:57 |
| mdbooth | mriedem: Right. I have neutron, but it didn't reproduce. | 13:57 |
| mdbooth | mriedem: I already have that. I have working neutron multinode. | 13:57 |
| mriedem | xenial? | 13:57 |
| mdbooth | However, it doesn't reproduce the issue. | 13:57 |
| *** ekuris has quit IRC | 13:57 | |
| *** roeyc has joined #openstack-nova | 13:57 | |
| mdbooth | Fedora 24 | 13:57 |
| mriedem | well, new enough libvirt | 13:57 |
| mriedem | ok | 13:57 |
| mriedem | PaulMurray: ^ | 13:57 |
| mriedem | nova meeting in 2 minutes | 13:57 |
| *** diogogmt_ has joined #openstack-nova | 13:58 | |
| mdbooth | I have also been able to force post-copy, but as I note I don't think that's relevant here | 13:58 |
| cdent | sdague: did you see my posting on the next steps of resource providers thread? | 13:58 |
| sdague | I guess not yet | 13:58 |
| mdbooth | Incidentally, post-copy is *fast* | 13:58 |
| cdent | there's debate over when to use list of dicts and when to use dict of dicts | 13:58 |
| PaulMurray | mriedem, mdbooth looking - what am I looking at ? | 13:59 |
| mriedem | mdbooth: does the size of the vm matter? to slow it down i mean | 13:59 |
| *** mingyu has quit IRC | 13:59 | |
| mdbooth | mriedem: So post-copy is completely orthogonal to the issue I'm having | 13:59 |
| mdbooth | The issue I'm having is that even when I prevent nova from doing the network switchover completely | 13:59 |
| mdbooth | The networking still works anyway | 13:59 |
| openstackgerrit | Diana Clarke proposed openstack/nova: Remove paramiko dependency https://review.openstack.org/367395 | 14:00 |
| mdbooth | I believe that's an artifact of a particular neutron implementation | 14:00 |
| cdent | sdague: basically from my perspective a dict of dicts is more flexible in terms of answer questions and also represents the constraints of the data more accurately, so I did that, because there was no spec. as the commit message implies: other opinions are valid and we need to figure it out | 14:00 |
| mriedem | PaulMurray: mdbooth is trying to recreate https://bugs.launchpad.net/nova/+bug/1605016 | 14:00 |
| mdbooth | I suspect there's some other neutron setup which does have the problem, but it's not the default. | 14:00 |
| *** diogogmt has quit IRC | 14:00 | |
| *** diogogmt_ is now known as diogogmt | 14:00 | |
| openstack | Launchpad bug 1605016 in OpenStack Compute (nova) "Post copy live migration interrupts network connectivity" [High,Confirmed] - Assigned to Matthew Booth (mbooth-9) | 14:00 |
| *** anush has joined #openstack-nova | 14:00 | |
| dansmith | mriedem: meeting time? or did I miss that it's canceled? | 14:00 |
| mriedem | it's on | 14:00 |
| sdague | cdent: ok, so where is the state of the debate. A list of dicts seems more sensible to me. But I'm not sure where things currently stand. | 14:00 |
| PaulMurray | mdbooth, dir will certainly have the problem | 14:01 |
| mdbooth | 'dir'? | 14:01 |
| cdent | if you read the message it has a link to some irc discussion last night, and then it tailed off because people had to leave. I made the posting because I needed to know how it resolved, but nothing yet | 14:01 |
| PaulMurray | s/dir/dvr/ | 14:01 |
| *** amotoki has quit IRC | 14:01 | |
| mdbooth | PaulMurray: So, I know essentially nothing about virtual networking | 14:01 |
| mdbooth | DVR is distributed virtual routing, right? | 14:02 |
| *** amotoki has joined #openstack-nova | 14:02 | |
| PaulMurray | mdbooth, yes, CVR is generally a switch in the middle | 14:02 |
| PaulMurray | mdbooth, DVR is each host doing its own routing | 14:02 |
| *** liangy has joined #openstack-nova | 14:02 | |
| mdbooth | My reproducer doesn't involve routing | 14:02 |
| mdbooth | So if the problem requires routing, then I haven't understood it | 14:03 |
| PaulMurray | mdbooth, so DVR needs the agents on each host to be reconfigured to route to the right host | 14:03 |
| *** MikeG451 has joined #openstack-nova | 14:03 | |
| mdbooth | But the guest still just sees a network segment, right? | 14:03 |
| PaulMurray | mdbooth, neutron will handle that, we just need to tell it to rebind the ports, but there might be some implementations that will work anyway | 14:03 |
| *** cmartin has joined #openstack-nova | 14:03 | |
| mdbooth | And it expects that the network segment remains the same throughout the migration process. | 14:04 |
| mdbooth | Is my test a valid test? | 14:04 |
| PaulMurray | Vas will do gap after migration so the ip address routing might be updated in some cases | 14:04 |
| PaulMurray | DVR will not | 14:04 |
| mdbooth | How would I configure DVR? | 14:04 |
| PaulMurray | good question | 14:04 |
| PaulMurray | sorry - reading back my laptop is auto-correcting every acronym I'm typing | 14:05 |
| *** burgerk has quit IRC | 14:05 | |
| mdbooth | PaulMurray: I just wrote an essay in the bug, btw :) | 14:05 |
| *** mingyu has joined #openstack-nova | 14:06 | |
| mdbooth | I think I know how to fix it, but I don't want to spend time on it until I've reproduced it. | 14:06 |
| mdbooth | Because otherwise I won't know if I've understood the problem, or if I've fixed it. | 14:06 |
| diana_clarke | I'd love to hear any thoughts you folks have on https://review.openstack.org/367395 | 14:06 |
| diana_clarke | (I'm currently trying to address the regression for Liberty & Mitaka, but also perhaps remove the paramiko dependency in time for Newton) | 14:07 |
| PaulMurray | mdbooth, if the networks work during post copy it implies there is no need to rebind the ports after the migration either | 14:07 |
| PaulMurray | which is probably not right - but I don't know enough neutron to be sure | 14:07 |
| mdbooth | I *suspect* the fact that the network works anyway is just an artifact of this backend. | 14:08 |
| mdbooth | But I'd like to see it not work with a different backend. | 14:08 |
| *** kfarr has joined #openstack-nova | 14:09 | |
| *** _coolsvap_ is now known as coolsvap | 14:09 | |
| *** tongli has joined #openstack-nova | 14:09 | |
| mdbooth | diana_clarke: Fun! | 14:10 |
| *** sdake has joined #openstack-nova | 14:10 | |
| *** roeyc has quit IRC | 14:11 | |
| *** sdake_ has quit IRC | 14:12 | |
| kashyap | mdbooth: dgilbert already has posted a potential reproducer info for you in response to your essay :-) | 14:12 |
| mdbooth | diana_clarke: So, paramiko 2.0 is fine, right? | 14:12 |
| diana_clarke | mdbooth: yes | 14:13 |
| *** su_zhang_ has quit IRC | 14:13 | |
| mdbooth | diana_clarke: And Newton has paramiko 2.0? | 14:13 |
| diana_clarke | mdbooth: yes | 14:13 |
| *** mingyu has quit IRC | 14:13 | |
| *** su_zhang has joined #openstack-nova | 14:13 | |
| mdbooth | So you really want this just for mitaka and liberty, because they have old paramiko? | 14:13 |
| mdbooth | Just trying to find out if I've understood | 14:14 |
| *** roeyc has joined #openstack-nova | 14:14 | |
| *** Guest19167 has quit IRC | 14:14 | |
| mdbooth | The suggestion: mitaka and liberty have broken paramiko keygen, make them use ssh-keygen instead. | 14:14 |
| mdbooth | Newton is fine, because paramiko 2 keygen is fine. | 14:15 |
| diana_clarke | mdbooth: Ideally, I would just remove the dependency entirely. | 14:15 |
| *** EinstCrazy has quit IRC | 14:15 | |
| diana_clarke | from liberty, mitaka, and newton | 14:15 |
| mdbooth | Oh, is that the only thing we use paramiko for? | 14:15 |
| diana_clarke | (but I'm open to other suggestions) | 14:15 |
| diana_clarke | mdbooth: yup | 14:16 |
| mdbooth | Wow | 14:16 |
| mdbooth | Yeah, we should totally ditch that imho | 14:16 |
| mdbooth | A parallel ssh implementation gives me the heebie jeebies anyway. | 14:16 |
| *** thorst has joined #openstack-nova | 14:16 | |
| mdbooth | I thought we were using custom crypto because it gave us some useful deep integration. | 14:17 |
| PaulMurray | mdbooth, https://wiki.openstack.org/wiki/Neutron/DVR/HowTo | 14:17 |
| *** amotoki has quit IRC | 14:17 | |
| mdbooth | PaulMurray: It starts with a video. I'm nervous already :) | 14:17 |
| mdbooth | Thanks, I'll read it. | 14:18 |
| mdbooth | (and watch) | 14:18 |
| *** su_zhang has quit IRC | 14:18 | |
| PaulMurray | get pop corn :) | 14:18 |
| PaulMurray | I haven't watched it yet | 14:18 |
| dansmith | jaypipes: cdent https://hangouts.google.com/call/z5va66cnczakredqf7m37oaqvie | 14:18 |
| mriedem | oh it's dvr related | 14:18 |
| mriedem | great | 14:18 |
| *** vishwanathj_zzz is now known as vishwanathj | 14:18 | |
| * cdent gets headphones | 14:18 | |
| *** zz_dimtruck is now known as dimtruck | 14:19 | |
| *** thorst_ has joined #openstack-nova | 14:19 | |
| dansmith | sdague: bauzas ^ | 14:19 |
| PaulMurray | mdbooth, if that doesn't work out I can find someone who can be helpful | 14:19 |
| mdbooth | PaulMurray: Thanks. | 14:19 |
| PaulMurray | most of dir was implemented by us | 14:19 |
| PaulMurray | (but not me) | 14:19 |
| *** EinstCrazy has joined #openstack-nova | 14:19 | |
| PaulMurray | DVR DVR - I hate auto-correct | 14:19 |
| jaypipes | bauzas, johnthetubaguy, dansmith, alex_xu, sdague, mriedem, cdent, alaski: https://hangouts.google.com/hangouts/_/pgy6ze6alnahldh2e546ldzk2ee | 14:20 |
| dansmith | jaypipes: we're all on the above hangout already | 14:20 |
| *** mlavalle has joined #openstack-nova | 14:20 | |
| *** zul has joined #openstack-nova | 14:20 | |
| mriedem | PaulMurray: does mdbooth also need this? https://review.openstack.org/#/c/275073/ | 14:21 |
| *** roeyc has quit IRC | 14:21 | |
| *** takashin has left #openstack-nova | 14:22 | |
| mdbooth | mriedem: I did notice that looked like a no-op, btw | 14:22 |
| *** thorst has quit IRC | 14:22 | |
| *** zul has quit IRC | 14:22 | |
| *** zul has joined #openstack-nova | 14:23 | |
| *** rlrossit_ has joined #openstack-nova | 14:23 | |
| mdbooth | mriedem: Although, unless it breaks it, it's not what I'm looking for. | 14:24 |
| *** sridhargaddam has joined #openstack-nova | 14:24 | |
| *** _ducttape_ has quit IRC | 14:24 | |
| mdbooth | mriedem: My current problem is that it works when I was hoping it was broken :) | 14:24 |
| *** bnemec has quit IRC | 14:24 | |
| *** ljxiash has joined #openstack-nova | 14:24 | |
| PaulMurray | mriedem, I don't think that is needed - its an optimisation | 14:25 |
| *** ihrachys has quit IRC | 14:25 | |
| PaulMurray | mriedem, hm, although in the end they should work together | 14:25 |
| PaulMurray | but for testing the problem it shouldn't be needed | 14:25 |
| *** rlrossit has quit IRC | 14:26 | |
| PaulMurray | mriedem, oh, I see, its in | 14:26 |
| *** sridharg has quit IRC | 14:27 | |
| *** ihrachys has joined #openstack-nova | 14:27 | |
| jroll | jaypipes: quick question, can mysql do alter column (specifically text -> longtext) without read locks these days? (and what are the chances people are actually using those versions?) | 14:27 |
| PaulMurray | mriedem, mdbooth the steps taken after migration now are the steps that should be taken when the post-copy switch happens | 14:27 |
| *** bnemec has joined #openstack-nova | 14:27 | |
| PaulMurray | so its largely the same - just need to know when to do it | 14:28 |
| *** amotoki has joined #openstack-nova | 14:28 | |
| *** nkrinner is now known as nkrinner_afk | 14:28 | |
| *** directxman12 has joined #openstack-nova | 14:29 | |
| *** moshele has quit IRC | 14:29 | |
| *** directxman12 has quit IRC | 14:29 | |
| *** sridhargaddam has quit IRC | 14:31 | |
| rpodolyaka | jroll: according to http://dev.mysql.com/doc/refman/5.7/en/innodb-create-index-overview.html concurrent reads should be ok | 14:32 |
| *** burgerk has joined #openstack-nova | 14:32 | |
| *** amotoki has quit IRC | 14:33 | |
| jroll | oh, interesting | 14:33 |
| jroll | 5.6+ | 14:33 |
| rpodolyaka | yeah, I think they improved a lot since 5.6 in terms of online DDL operations | 14:33 |
| *** rodrigods has quit IRC | 14:35 | |
| *** rodrigods has joined #openstack-nova | 14:35 | |
| *** scarab_ has joined #openstack-nova | 14:37 | |
| *** salv-orlando has joined #openstack-nova | 14:40 | |
| mriedem | burgerk: how are things looking on https://review.openstack.org/#/c/366955/ ? | 14:41 |
| *** shangxdy has joined #openstack-nova | 14:41 | |
| *** EinstCrazy has quit IRC | 14:41 | |
| * sfinucan comes up from a refactor that took way too long | 14:42 | |
| burgerk | mriedem: plan to have new patch up in about an hour | 14:42 |
| *** pwnall has joined #openstack-nova | 14:42 | |
| *** mvk has quit IRC | 14:42 | |
| mriedem | ok | 14:42 |
| *** pwnall is now known as Guest34723 | 14:42 | |
| * sfinucan has realized how many things can go wrong when scheduling an instance with NUMA topo and is amazed it works at all | 14:43 | |
| *** baoli has joined #openstack-nova | 14:43 | |
| *** sridhargaddam has joined #openstack-nova | 14:44 | |
| *** mars_toktonaliev has quit IRC | 14:45 | |
| *** zenoway has joined #openstack-nova | 14:45 | |
| mdbooth | sfinucan: LOL, don't touch it! | 14:46 |
| sfinucan | mdbooth: Just wait til you see this exception list in a doctring | 14:46 |
| sfinucan | ...and I'm almost sure I missed some :) | 14:46 |
| *** armax has joined #openstack-nova | 14:49 | |
| cdent | dansmith, jaypipes: I just though of another reason why a list of anon dicts is sometimes good: if the result is going to be long (say all the allocations on a shared resource provider) a list is less complex to limit/filter/sort | 14:50 |
| sdague | dansmith: oh, one other thing that exposes with the logging patch - https://review.openstack.org/#/c/366186/ - we never have disk allocations | 14:51 |
| dansmith | sdague: right | 14:52 |
| dansmith | sdague: because we have zero-disk flavors right? | 14:52 |
| sdague | yes | 14:52 |
| dansmith | in the gate | 14:52 |
| dansmith | locally I definitely see disk allocations | 14:52 |
| cfriesen | sfinucan: you should have seen it back in Havana. :) | 14:52 |
| sdague | should we just flag that as a bug to fix later? | 14:52 |
| *** dikonoor has quit IRC | 14:52 | |
| dansmith | sdague: it's not a bug | 14:52 |
| dansmith | sdague: we have no disk, thus no disk allocation | 14:52 |
| sdague | zero-disk flavors still take up disk | 14:53 |
| dansmith | sdague: .. you mean because of the config file? :) | 14:53 |
| *** baoli has quit IRC | 14:53 | |
| sdague | no, because that just means autosize to glance image size | 14:53 |
| *** baoli has joined #openstack-nova | 14:54 | |
| dansmith | sdague: we've never tracked that, AFAIK, and the whole point of this is to not charge instances with no local disk against the compute node | 14:54 |
| *** marst has joined #openstack-nova | 14:55 | |
| sdague | dansmith: ok, but no local disk is different, and not indicated by disk == 0 | 14:55 |
| dansmith | sdague: zero root disk instances in the real world are volume-backed and thus don't have any local residue other than image caches, right? | 14:55 |
| sdague | we just added a new api to really mean, don't give me a local disk | 14:55 |
| *** abalutoiu has joined #openstack-nova | 14:55 | |
| sdague | dansmith: they don't have to be | 14:55 |
| dansmith | sdague: oh you mean when an ephemeral is root=0, which means image size, I see what you mean | 14:56 |
| sdague | yes | 14:56 |
| dansmith | and you said that sorry.. how does the RT deal with that now? | 14:56 |
| sdague | I don't know | 14:56 |
| dansmith | because I kinda expect it does what we do here which is count it as zero | 14:56 |
| *** Guest34723 has quit IRC | 14:56 | |
| sdague | jaypipes / bauzas do you know the current RT handling here? | 14:57 |
| dansmith | sdague: I dont see that it is handled at all in RT | 14:59 |
| dansmith | counted as zero AFAICT | 14:59 |
| *** gouthamr has joined #openstack-nova | 14:59 | |
| *** jraju has quit IRC | 14:59 | |
| *** TravT has joined #openstack-nova | 15:00 | |
| *** rlrossit_ has quit IRC | 15:00 | |
| *** gouthamr_ has joined #openstack-nova | 15:00 | |
| dansmith | sdague: https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L972-L976 | 15:00 |
| openstackgerrit | Jay Pipes proposed openstack/nova-specs: Amend placement API with correct PUT inventory https://review.openstack.org/366888 | 15:00 |
| jaypipes | cdent, dansmith: just running all tests on the code. | 15:01 |
| dansmith | jaypipes: thanks | 15:01 |
| *** aloga_ has quit IRC | 15:02 | |
| *** rlrossit_ has joined #openstack-nova | 15:03 | |
| cdent | jaypipes, dansmith: put allocation is currently a list of doubly nested dicts:http://specs.openstack.org/openstack/nova-specs/specs/newton/approved/generic-resource-pools.html#put-allocations-consumer-uuid | 15:03 |
| cdent | that's not very useful as a representation on the response | 15:03 |
| *** rlrossit_ has quit IRC | 15:04 | |
| cdent | I'll do what I think is right and then we can talk about it | 15:04 |
| *** gouthamr has quit IRC | 15:04 | |
| *** rlrossit_ has joined #openstack-nova | 15:04 | |
| *** akshai_ has quit IRC | 15:04 | |
| burgerk | mriedem: for test_migrations functional test, I would need to add the row during the previous test (check_019) then assert type change and row remains in check_020? | 15:05 |
| *** READ10 has joined #openstack-nova | 15:05 | |
| mriedem | burgerk: the _check_020 runs after the migration is applied | 15:05 |
| mriedem | there is a pre-migration method to populate the data | 15:05 |
| mriedem | burgerk: called _pre_upgrade_020 | 15:06 |
| mriedem | burgerk: e.g. https://github.com/openstack/nova/blob/master/nova/tests/unit/db/test_migrations.py#L645 | 15:06 |
| alaski | burgerk: you can look at tests/unit/db/test_migrations.py to see how it's done for the other db which has some tests that load data | 15:06 |
| burgerk | ok that is what I was missing | 15:06 |
| *** jlanoux has quit IRC | 15:06 | |
| burgerk | mriedem: alaski: thanks | 15:08 |
| *** sridhargaddam has quit IRC | 15:08 | |
| *** scarab_ has quit IRC | 15:10 | |
| *** su_zhang has joined #openstack-nova | 15:12 | |
| *** akshai has joined #openstack-nova | 15:14 | |
| *** prateek has joined #openstack-nova | 15:14 | |
| *** raunak has joined #openstack-nova | 15:14 | |
| *** ducttape_ has joined #openstack-nova | 15:15 | |
| mdbooth | In neutron, what is dhcp associated with? | 15:15 |
| mdbooth | A network, a subnet, a router | 15:15 |
| ZZelle | mdbooth, a network | 15:16 |
| *** kevinz has quit IRC | 15:16 | |
| ZZelle | mdbooth, oups not exactly | 15:16 |
| ZZelle | mdbooth, a network is associated to dhcp-agents and you configure dhcp per subnet | 15:16 |
| openstackgerrit | Matt Riedemann proposed openstack/nova: virt: handle unicode when logging LifecycleEvents https://review.openstack.org/367455 | 15:16 |
| mriedem | easy fix ^ lifecycle events in the virt layer are broken if you're not using english locales | 15:17 |
| mdbooth | ZZelle: If I update public-subnet to set enable_dhcp=True, will the neutron fairy make magic? | 15:17 |
| ZZelle | mdbooth, yes or at least it should :) | 15:18 |
| mdbooth | mriedem: Sorry, that appears to be an issue with your language provider. Please select correct heritage and retry. | 15:18 |
| mriedem | heh | 15:18 |
| *** akshai has quit IRC | 15:19 | |
| *** akshai has joined #openstack-nova | 15:19 | |
| *** prateek has quit IRC | 15:20 | |
| mdbooth | ZZelle: have invoked neutron fairy :) | 15:20 |
| * mdbooth claps his hands furiously | 15:20 | |
| jaypipes | dansmith, cdent: k, done with meeting. had a few func test failures. fixing those up now. shouldn't be more than half hour to push. thx for patience. | 15:21 |
| *** sdake_ has joined #openstack-nova | 15:21 | |
| dansmith | jaypipes: cool, thanks | 15:21 |
| *** abalutoiu has quit IRC | 15:21 | |
| *** adisky has quit IRC | 15:21 | |
| *** sdake has quit IRC | 15:22 | |
| *** rrecio has joined #openstack-nova | 15:22 | |
| openstackgerrit | Diana Clarke proposed openstack/nova: Remove paramiko dependency https://review.openstack.org/367395 | 15:23 |
| *** mjura has quit IRC | 15:24 | |
| openstackgerrit | Mikhail Feoktistov proposed openstack/nova: Check hypervisor version while getting bus type https://review.openstack.org/282398 | 15:24 |
| *** rloo has joined #openstack-nova | 15:25 | |
| *** annegentle has joined #openstack-nova | 15:27 | |
| sdague | mriedem: +2 | 15:27 |
| *** amotoki has joined #openstack-nova | 15:29 | |
| *** mriedem has quit IRC | 15:29 | |
| *** zenoway has quit IRC | 15:30 | |
| rloo | hi, ironic is adding notifications. i was looking at nova notifications (https://wiki.openstack.org/wiki/SystemUsageData) and wondering what your guidelines were wrt event types. | 15:30 |
| rloo | why do some have phases start/end, or start/end/error, and others don't? | 15:31 |
| *** hshiina has quit IRC | 15:31 | |
| mdbooth | PaulMurray: That HowTo says: | 15:31 |
| mdbooth | "Be sure that your firewall or security groups allows UDP traffic over the vxlan port to pass between the hosts" | 15:31 |
| mdbooth | What does that mean? | 15:32 |
| mdbooth | I ask in the context that I have to instances on the same network but different hosts which can't ping each other. | 15:32 |
| mdbooth | Although I haven't done the security-group dance, yet | 15:32 |
| mdbooth | Is that required? | 15:32 |
| *** akshai has quit IRC | 15:32 | |
| *** akshai has joined #openstack-nova | 15:33 | |
| *** amotoki has quit IRC | 15:34 | |
| *** jlanoux has joined #openstack-nova | 15:35 | |
| *** matrohon has quit IRC | 15:37 | |
| PaulMurray | carl_baldwin, are you around? | 15:37 |
| carl_baldwin | PaulMurray: indeed I am | 15:37 |
| *** mgoddard_ has joined #openstack-nova | 15:37 | |
| PaulMurray | mdbooth, is trying to set up a DVR devstack - any advice ? | 15:37 |
| PaulMurray | carl_baldwin, ^^ | 15:38 |
| *** zenoway has joined #openstack-nova | 15:39 | |
| *** zul has quit IRC | 15:39 | |
| haleyb | mdbooth: you can have a single-node dvr setup in devstack by putting Q_DVR_MODE=dvr_snat in local.conf. A multi-node setup is a little harder, but doable | 15:39 |
| *** jvgrant has joined #openstack-nova | 15:40 | |
| *** zul has joined #openstack-nova | 15:40 | |
| *** mgoddard has quit IRC | 15:40 | |
| haleyb | that setting will trigger all the namespaces to be configured | 15:40 |
| *** zul has quit IRC | 15:40 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova: virt: Split out 'numa' and 'compat' modules https://review.openstack.org/367467 | 15:41 |
| openstackgerrit | Stephen Finucane proposed openstack/nova: numa: Rework docstrings https://review.openstack.org/367468 | 15:41 |
| openstackgerrit | Stephen Finucane proposed openstack/nova: numa: Standardize '_get_XXX_constraints' functions https://review.openstack.org/367469 | 15:41 |
| openstackgerrit | Stephen Finucane proposed openstack/nova: numa: Flatten functions https://review.openstack.org/367470 | 15:41 |
| *** hemnafk is now known as hemna | 15:41 | |
| *** zul has joined #openstack-nova | 15:41 | |
| PaulMurray | mdbooth, is there a default security group ? | 15:41 |
| carl_baldwin | PaulMurray: mdbooth: looks like haleyb is a little quicker than I am. You're in good hands. I'll keep an eye out here in case I can add anything. | 15:41 |
| PaulMurray | carl_baldwin, thanks | 15:41 |
| *** andreas_s has quit IRC | 15:42 | |
| *** READ10 has quit IRC | 15:42 | |
| haleyb | carl_baldwin: i added an irc filter to highlight 'dvr' today, so now see it's all over the place | 15:42 |
| *** gouthamr_ is now known as gouthamr | 15:42 | |
| PaulMurray | haleyb, that's because everyone loves it so much | 15:43 |
| *** _ducttape_ has joined #openstack-nova | 15:44 | |
| *** browne has joined #openstack-nova | 15:44 | |
| * haleyb can feel the sarcasm all the way across the ocean :) | 15:44 | |
| *** zenoway has quit IRC | 15:44 | |
| *** ducttape_ has quit IRC | 15:44 | |
| *** mgoddard_ has quit IRC | 15:44 | |
| sdague | hmmm... what's up with - No instance_mapping found for instance 0edb43c0-b007-40eb-a9eb-7a294cb9de57. This should not be happening and will lead to errors in the future. Please open a bug at https://bugs.launchpad.net/nova | 15:44 |
| sdague | happing hundreds of times in every gate run? | 15:44 |
| jaypipes | alaski: ^ | 15:45 |
| carl_baldwin | haleyb: I wondered how you were all over that. | 15:46 |
| sdague | http://logs.openstack.org/18/361518/9/gate/gate-tempest-dsvm-neutron-full-ubuntu-xenial/fb629d7/logs/screen-n-api.txt.gz?level=TRACE#_2016-09-07_07_18_15_878 as an example | 15:46 |
| bauzas | sdague: I had the same problem with my local devstack, I thought it was a local conf issue | 15:47 |
| dansmith | sdague: see the dbdeadlock right below that? | 15:48 |
| sdague | dansmith: yeh, but that's unrelated | 15:48 |
| dansmith | sdague: you know that? | 15:48 |
| sdague | this happens 2400 times | 15:48 |
| dansmith | I mean aside from that it's once and the other is a lot | 15:48 |
| dansmith | it's kinda in the same path | 15:49 |
| sdague | yes, the db deadlock is related to the get me a network issue | 15:49 |
| openstackgerrit | Diana Clarke proposed openstack/nova: Remove paramiko dependency https://review.openstack.org/367395 | 15:49 |
| sdague | dansmith: ok, there is 1 deadlock on one test | 15:49 |
| *** pwnall has joined #openstack-nova | 15:49 | |
| sdague | there are 2400+ of this warning, happening on basically every test | 15:49 |
| dansmith | yeah, I get that ;) | 15:50 |
| *** pwnall is now known as Guest50218 | 15:50 | |
| dansmith | is there an issue open for the deadlock? | 15:50 |
| diana_clarke | sdague, mdbooth: Thanks for the reviews! I added a bug # in the latest revision ^^^ | 15:50 |
| sdague | dansmith: there is not yet, I was just looking at one of the uncategorized fails | 15:50 |
| sdague | and noticed this other issue | 15:50 |
| dansmith | sdague: just curious based on the log discussion earlier | 15:50 |
| *** mriedem has joined #openstack-nova | 15:51 | |
| *** jgriffith_away is now known as jgriffith | 15:53 | |
| mriedem | dims: you worked on a lot of the pycrypto/cryptodome/paramiko stuff, so you might want to take a look at https://review.openstack.org/#/c/367395/ | 15:53 |
| dansmith | mriedem: you know anything about this get-me-a-network deadlock thing? | 15:53 |
| mriedem | huh? | 15:54 |
| dansmith | mriedem: sdague says this is get-me-a-network related: http://logs.openstack.org/18/361518/9/gate/gate-tempest-dsvm-neutron-full-ubuntu-xenial/fb629d7/logs/screen-n-api.txt.gz?level=TRACE#_2016-09-07_07_18_16_681 | 15:54 |
| mriedem | no | 15:54 |
| mriedem | it's not | 15:54 |
| dansmith | looks like it's actually a deadlock in the rollback of whatever the real error is maybe? | 15:55 |
| *** maestropandy has quit IRC | 15:55 | |
| mriedem | it's an old bug i'm trying to find the link | 15:55 |
| *** READ10 has joined #openstack-nova | 15:55 | |
| mriedem | https://bugs.launchpad.net/nova/+bug/1480305 | 15:56 |
| openstack | Launchpad bug 1480305 in OpenStack Compute (nova) "DBDeadlock inserting into instance_extra" [Medium,Triaged] | 15:56 |
| mriedem | our e-r query is stale | 15:56 |
| mriedem | http://status.openstack.org//elastic-recheck/index.html#1480305 | 15:56 |
| dansmith | oh, nm that's in a finally block | 15:56 |
| dims | mriedem : looking | 15:57 |
| *** BjoernT has joined #openstack-nova | 15:57 | |
| mriedem | http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22ec2_instance_create%5C%22%20AND%20message%3A%5C%22DBDeadlock%5C%22%20AND%20message%3A%5C%22u'INSERT%20INTO%20instance_extra%5C%22%20AND%20tags%3A%5C%22screen-n-api.txt%5C%22%20AND%20voting%3A1&from=7d | 15:57 |
| mriedem | i'll fix the query | 15:58 |
| *** akshai has quit IRC | 15:58 | |
| *** mgoddard has joined #openstack-nova | 15:58 | |
| *** rlrossit_ has quit IRC | 15:59 | |
| *** akshai has joined #openstack-nova | 15:59 | |
| *** sayalilunkad has quit IRC | 16:00 | |
| mriedem | dansmith: there was a quick fix attempt here https://review.openstack.org/#/c/292800/ | 16:00 |
| mriedem | i suggested doing the instance-related table inserts in separate transactions | 16:01 |
| mriedem | although the retry decorator would be simple | 16:02 |
| *** su_zhang has quit IRC | 16:02 | |
| *** su_zhang has joined #openstack-nova | 16:03 | |
| dansmith | I kinda think they should be in the same transaction | 16:03 |
| dansmith | we assume that they're both there in a lot of code now I think, so we wouldn't really want one to succeed and one to fail | 16:04 |
| *** paul-carlton has joined #openstack-nova | 16:04 | |
| mriedem | since no one else is working on it, i'm fine with just humping in the retry decorator | 16:06 |
| dansmith | yeah | 16:06 |
| openstackgerrit | Matt Riedemann proposed openstack/nova: db: retry on deadlocks while adding an instance https://review.openstack.org/292800 | 16:06 |
| dansmith | I think that's probably the best option here although it always feels like a hack | 16:06 |
| mriedem | +W if you want, i just +2ed and rebased | 16:07 |
| *** su_zhang has quit IRC | 16:07 | |
| *** liverpooler has joined #openstack-nova | 16:08 | |
| dansmith | mriedem: got it | 16:09 |
| diana_clarke | I don't think I have the ability to add milestones to this bug: https://bugs.launchpad.net/nova/+bug/1621536 | 16:11 |
| openstack | Launchpad bug 1621536 in OpenStack Compute (nova) "Remove paramiko dependency" [Undecided,In progress] - Assigned to Diana Clarke (diana-clarke) | 16:11 |
| mriedem | diana_clarke: which milestone would you target? | 16:11 |
| diana_clarke | (I'd like it to be considered for Newton still, if possible) | 16:11 |
| mriedem | diana_clarke: doesn't seem justified for newton | 16:12 |
| diana_clarke | mriedem: Newton, Mitaka, and Liberty | 16:12 |
| openstackgerrit | Chris Dent proposed openstack/nova: [placement] add two ways to GET allocations https://review.openstack.org/366789 | 16:12 |
| mriedem | diana_clarke: backporting that is going to be not fun | 16:12 |
| *** zenoway has joined #openstack-nova | 16:12 | |
| cdent | dansmith, jaypipes I've got some concerns on that ^ (Noted in commit message and todos within the diff) | 16:12 |
| diana_clarke | mriedem: packaging or code-wise? | 16:12 |
| cdent | will update the etherpad | 16:12 |
| mriedem | diana_clarke: code | 16:13 |
| dansmith | mriedem: yeah, sdague said not for newton too and I agree | 16:13 |
| dansmith | cdent: okay | 16:13 |
| *** ljxiash has quit IRC | 16:13 | |
| diana_clarke | mriedem: it's pretty easy code-wise | 16:13 |
| mriedem | diana_clarke: it's a pretty non-trivial change for something like stable/liberty at this point | 16:13 |
| mriedem | and i don't see the justification | 16:14 |
| *** burt has joined #openstack-nova | 16:14 | |
| diana_clarke | dansmith, mriedem: ah, I misunderstood sdague then. I thought he was saying: "add a bug, so we can consider it for newton" | 16:14 |
| artom | Asking here because brain is mostly fried from awful night - is there a specific exception that oslo messaging raises if the recipient is at a lower RPC version than the sender? | 16:14 |
| mriedem | the paramiko/pycrypt/cryptodome stuff has been a hairy mess which is why i'd like to avoid a lot of that complexity right now | 16:14 |
| dansmith | diana_clarke: he said hold until master re-opens, which means not newton, wait for ocata | 16:14 |
| artom | I would be very grateful for at least pointers to where to look in the code :) | 16:14 |
| alaski | sdague: looking into it | 16:15 |
| mriedem | artom: https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/exceptions.py ? | 16:15 |
| artom | Ex: I send 1.13 but the recipient hasn't been upgraded yet is only understands 1.12 | 16:15 |
| dansmith | artom: you only get that on a call (not cast) and that should never actually happen because of the pin | 16:15 |
| mriedem | there are others in here https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/rpc/client.py | 16:16 |
| artom | mriedem, thanks :) It would be InvalidTarget then? "certain pre-conditions" is pretty vague :/ | 16:16 |
| dansmith | artom: so we don't really catch it anywhere since it shouldn't be a thing (and doesn't work for cast) | 16:16 |
| diana_clarke | mriedem, sdague, dansmith: okie dokie. I'll just work on it downstream then. | 16:16 |
| *** zenoway has quit IRC | 16:16 | |
| artom | dansmith, is pinning an automatic thing then? | 16:17 |
| dansmith | artom: there is an 'auto' mode now, but in general it's a manual thing | 16:17 |
| dansmith | artom: and it's just kinda your responsibility if you run mixed versions to have set the pin properly.. always has been | 16:17 |
| dansmith | artom: my point is, I don't think there's any prior art for trying to handle the InvalidTarget automatically because it can't be universally applied and because the way we handle compatibility is via the pin | 16:18 |
| dansmith | cdent: which concerns exactly? just the sort/filter limiting? | 16:18 |
| artom | dansmith, context here is our downstream backports of the dhcp_release thing (I guess maybe move this to our internal channel?) | 16:18 |
| dansmith | artom: right, so backports of such things are really really hard to get right, which is why we try to never do them :) | 16:19 |
| cdent | dansmith: sort/filter and lack of self describing: the output from /reso..allo and /alloca... look nearly the same but the key of the dict is an entirely different thing. Do we care? | 16:19 |
| dansmith | artom: is that a call or a cast? I thought the latter | 16:19 |
| artom | dansmith, it's a call | 16:19 |
| dansmith | cdent: they're different resources so I don't really see why it matters | 16:19 |
| dansmith | artom: okay, well, anyway, with my upstream hat on "that's a problem for you to solve downstream, so good luck with that" :P | 16:19 |
| *** diana_clarke has left #openstack-nova | 16:20 | |
| *** sahid has quit IRC | 16:20 | |
| cdent | dansmith: i'm cool with that | 16:21 |
| *** tesseract- has quit IRC | 16:21 | |
| dansmith | cdent: okay | 16:21 |
| *** diana_clarke has joined #openstack-nova | 16:21 | |
| cdent | dansmith: so the other conflict is PUT allocations looks nothing like GET allocations | 16:22 |
| *** diana_clarke has left #openstack-nova | 16:22 | |
| cdent | (I put that on the etherpad) | 16:22 |
| *** markus_z has quit IRC | 16:22 | |
| dansmith | cdent: ugh.. pretty uncool that we got so sideways on this :/ | 16:22 |
| *** thorst_ has quit IRC | 16:23 | |
| *** nic has joined #openstack-nova | 16:23 | |
| *** diana_clarke has joined #openstack-nova | 16:23 | |
| *** unicell has joined #openstack-nova | 16:23 | |
| *** thorst has joined #openstack-nova | 16:23 | |
| pumaranikar | does anyone know how nova logging setup behave with testr. | 16:24 |
| *** kaisers has quit IRC | 16:24 | |
| pumaranikar | is this the place https://github.com/openstack/nova/blob/master/nova/tests/fixtures.py#L87 | 16:24 |
| cdent | dansmith: I have no idea what happened in general, but for me it was simply review fatigue and naivety on the extent to which we would iterate during the cycle (rather than at the end). I just kind of assumed we would make a lot of changes throughout the cycle in a paced fashion and that the details were effectively stubs | 16:24 |
| dansmith | yeah, just commenting not blaming :) | 16:25 |
| *** unicell1 has quit IRC | 16:25 | |
| diana_clarke | mriedem: re: "the paramiko/pycrypt/cryptodome stuff has been a hairy mess". Isn't that an argument *for* removing this dependency before shipping newton? | 16:25 |
| mriedem | diana_clarke: against | 16:25 |
| cdent | and I'm not excusing, just conversing | 16:25 |
| * cdent IS SO LONELY :) | 16:26 | |
| dansmith | lol | 16:26 |
| *** praveen has joined #openstack-nova | 16:26 | |
| mriedem | diana_clarke: as in, i don't like touching something that has had a lot of complicated churn in the past 1 week from RC1 if it's not a critical bug | 16:26 |
| dansmith | cdent: M-x eliza ? | 16:26 |
| * cdent shuns dansmith | 16:26 | |
| dansmith | er, M-x doctor | 16:27 |
| diana_clarke | mriedem: but, but... it's a revert back to the state before all that churn. | 16:27 |
| * dansmith couldn't recall it from cold storage quick enough | 16:27 | |
| diana_clarke | mriedem: anyhoo, I'll move on ;) | 16:27 |
| cdent | no dansmith, I'm shunning emacs | 16:27 |
| * cdent gives a cookie to diana_clarke | 16:27 | |
| *** browne has left #openstack-nova | 16:27 | |
| diana_clarke | cdent: thanks, much needed ;) | 16:28 |
| cdent | now I want one | 16:28 |
| * cdent has run out of cookies | 16:28 | |
| *** cher75 has joined #openstack-nova | 16:28 | |
| *** timello has quit IRC | 16:29 | |
| alaski | sdague: that warning message shows up because the cells setup doesn't happen in the gate. But since we need to assume that deployments won't upgrade immediately in N it'll be best to remove that. | 16:29 |
| danpb | diana_clarke: if the nova code switch from ssh -> paramiko had been something we'd done in the last few months, then it would be a reason to consider reverting for this release | 16:29 |
| *** rama_y has joined #openstack-nova | 16:29 | |
| danpb | diana_clarke: but we've had this paramiko code in nova since Liberty now, so its hard to make a case that its critical to revert at this point in the release cycle | 16:29 |
| *** dimtruck is now known as zz_dimtruck | 16:30 | |
| openstackgerrit | Gábor Antal proposed openstack/nova: Throw exception if numa_nodes is not set to integer greater than 0 https://review.openstack.org/337380 | 16:30 |
| openstackgerrit | Eric Brown proposed openstack/nova: Fix config option types https://review.openstack.org/168481 | 16:31 |
| jaypipes | cdent: see comment on commit messagte | 16:31 |
| *** amotoki has joined #openstack-nova | 16:31 | |
| *** amotoki has quit IRC | 16:31 | |
| openstackgerrit | Eric Brown proposed openstack/nova: VMware: Enforce minimum vCenter version of 5.5 https://review.openstack.org/253666 | 16:31 |
| *** amotoki has joined #openstack-nova | 16:31 | |
| openstackgerrit | Eric Brown proposed openstack/nova: Update man pages with latest version and dates https://review.openstack.org/325441 | 16:31 |
| *** thorst has quit IRC | 16:31 | |
| *** ametts has joined #openstack-nova | 16:32 | |
| *** browne has joined #openstack-nova | 16:32 | |
| diana_clarke | danpb: yup, it's just a shame that adoption lags so far behind that by time we get these error reports it's too late... anyhoo, I'll play in the downstream sandbox instead ;) | 16:33 |
| cdent | thanks jaypipes | 16:33 |
| cdent | dansmith: how you feel about jaypipes's suggestion on https://review.openstack.org/#/c/366789/ | 16:33 |
| *** zz_dimtruck is now known as dimtruck | 16:34 | |
| *** READ10 has quit IRC | 16:34 | |
| dansmith | cdent: I shall comment | 16:35 |
| cdent | gracias | 16:35 |
| * cdent is in two meetings, so I'll wait to do anything for a while | 16:35 | |
| *** fragatina has joined #openstack-nova | 16:37 | |
| jaypipes | dansmith, cdent: cool to remove the top-level 'resource_providers' key sure | 16:37 |
| cdent | standard to have some kind of top level key so we can later add a 'links' key or some such | 16:37 |
| * cdent shrugs | 16:37 | |
| dansmith | cdent: oh I thought this was under allocations | 16:38 |
| dansmith | cdent: top level of allocations makes sense to me | 16:38 |
| cdent | dansmith: can you comment for sake of memory, otherwise I'll forget | 16:38 |
| *** jlanoux has quit IRC | 16:38 | |
| cdent | please and thank you | 16:38 |
| dansmith | cdent: I did | 16:39 |
| cdent | dansmith: i mean about which top level you want | 16:39 |
| jaypipes | so we'd have two 'allocations' keys? | 16:40 |
| auggy | diana_clarke: thanks for looking into the paramiko issue. we've had a number of bugs reported for things related to this, i'll have to dig them up to find out how we resolved them | 16:40 |
| *** tongli_ has joined #openstack-nova | 16:41 | |
| *** fragatina has quit IRC | 16:41 | |
| auggy | diana_clarke: https://bugs.launchpad.net/nova/+bug/1483132 | 16:42 |
| openstack | Launchpad bug 1483132 in OpenStack Compute (nova) "ssh-keygen-to-Paramiko change breaks third-party tools" [Low,Fix released] - Assigned to Corey Wright (coreywright) | 16:42 |
| *** timello has joined #openstack-nova | 16:42 | |
| dansmith | auggy: did you see the commit message? lots of links in there, including that one I think | 16:43 |
| *** clenimar has joined #openstack-nova | 16:43 | |
| auggy | dansmith: ah ok no i didn't, sorry | 16:43 |
| *** raunak has quit IRC | 16:43 | |
| *** tongli has quit IRC | 16:44 | |
| auggy | dansmith: oh geez it helps if i actually read the bug :/ | 16:44 |
| auggy | diana_clarke: sorry for the redundancy there :) | 16:45 |
| *** cher75 has quit IRC | 16:45 | |
| jaypipes | cdent, dansmith: like cdent says, it's probably good to keep 'resource_providers' as top-level key in the response. in future, I can see adding a links collection with links to "filtered" results, etc | 16:45 |
| diana_clarke | auggy: no worries :) | 16:45 |
| dansmith | jaypipes: I'm cool with a toplevel, but GET /allocations with a resource_providers toplevel makes no sense to me | 16:45 |
| *** tongli_ has quit IRC | 16:46 | |
| jaypipes | dansmith: the alternative is to have the resource provider generation repeated for each allocation record and use anon dicts. | 16:46 |
| *** code-R_ has quit IRC | 16:47 | |
| *** rlrossit has joined #openstack-nova | 16:47 | |
| dansmith | jaypipes: I don't see what the toplevel has to do with list of anon dicts or not being able to do the generation | 16:47 |
| *** lucasagomes is now known as lucas-dinner | 16:48 | |
| jaypipes | dansmith: where would you put the resource provider generation in cdent's proposed response format? | 16:49 |
| *** derekh has quit IRC | 16:49 | |
| dansmith | jaypipes: I see, you added yet another level underneath the provider uuid for allocations | 16:49 |
| jaypipes | yeh | 16:50 |
| dansmith | I guess I would call that resources: | 16:50 |
| dansmith | whatever | 16:50 |
| dansmith | just seems weird to me to ask for allocations and get back resource_providers | 16:50 |
| jaypipes | sure, that would work | 16:50 |
| openstackgerrit | Andrew Laski proposed openstack/nova: Remove misleading warning message https://review.openstack.org/367510 | 16:52 |
| alaski | sdague: ^ | 16:52 |
| cdent | dansmith, jaypipes I'll try to digest this into something that makes sense. if you guys have a concrete (and new) proposal, please put it on the review. otherwise I'll chew on what's been discussed and poop out some goodness | 16:52 |
| jaypipes | cdent: k | 16:53 |
| cdent | the main goal is: get the generation in there, right? | 16:53 |
| *** Apoorva has joined #openstack-nova | 16:55 | |
| *** devananda|travel is now known as devananda | 16:56 | |
| *** lpetrut has joined #openstack-nova | 16:56 | |
| *** jheroux has quit IRC | 16:57 | |
| *** danpb has quit IRC | 17:00 | |
| *** anush has quit IRC | 17:00 | |
| *** diana_clarke has left #openstack-nova | 17:00 | |
| *** jaosorior has quit IRC | 17:01 | |
| *** unicell has quit IRC | 17:01 | |
| *** ducttape_ has joined #openstack-nova | 17:01 | |
| *** swamireddy has quit IRC | 17:03 | |
| *** migi has joined #openstack-nova | 17:03 | |
| *** paul-carlton has quit IRC | 17:03 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova: Add deprecated_reason for use_usb_tablet option https://review.openstack.org/367517 | 17:03 |
| mriedem | bauzas: johnthetubaguy: ^ | 17:04 |
| mriedem | we had a reno for that but the help text didn't make it clear | 17:04 |
| *** tongli has joined #openstack-nova | 17:04 | |
| *** _ducttape_ has quit IRC | 17:04 | |
| openstackgerrit | Matt Riedemann proposed openstack/nova: Add deprecated_reason for use_usb_tablet option https://review.openstack.org/367517 | 17:05 |
| *** paul-carlton has joined #openstack-nova | 17:05 | |
| *** tongli has quit IRC | 17:08 | |
| *** zul has quit IRC | 17:10 | |
| *** ducttape_ has quit IRC | 17:12 | |
| *** cmartin_ has joined #openstack-nova | 17:13 | |
| *** ducttape_ has joined #openstack-nova | 17:13 | |
| cmartin_ | Could anybody review my commit? https://review.openstack.org/#/c/364337/ | 17:13 |
| *** zul has joined #openstack-nova | 17:14 | |
| *** su_zhang has joined #openstack-nova | 17:15 | |
| *** sudipto has joined #openstack-nova | 17:16 | |
| *** Guest50218 has quit IRC | 17:16 | |
| *** sudipto_ has joined #openstack-nova | 17:16 | |
| *** rlrossit has quit IRC | 17:17 | |
| openstackgerrit | Kenneth Burger proposed openstack/nova: Increase BDM column in build_requests table https://review.openstack.org/366955 | 17:18 |
| *** rlrossit has joined #openstack-nova | 17:18 | |
| mriedem | dansmith: thanks, i was just commenting | 17:19 |
| mriedem | i had the feeling this was whack a mole | 17:19 |
| dansmith | mriedem: np | 17:19 |
| *** rlrossit has quit IRC | 17:22 | |
| mriedem | gibi: fyi i'm seeing these in gate runs http://logs.openstack.org/55/367455/1/check/gate-tempest-dsvm-cells/1d8af83/logs/screen-n-api.txt.gz?level=TRACE#_2016-09-08_15_53_14_619 | 17:23 |
| mriedem | not sure what needs to be set to get rid of that warning | 17:23 |
| *** pwnall has joined #openstack-nova | 17:23 | |
| *** pwnall is now known as Guest83073 | 17:24 | |
| mriedem | alaski: were you talking to sdague earlier about removing this warning? http://logs.openstack.org/55/367455/1/check/gate-tempest-dsvm-cells/1d8af83/logs/screen-n-api.txt.gz?level=TRACE#_2016-09-08_15_58_30_538 | 17:24 |
| mriedem | b/c it's all over the cells v1 job logs | 17:24 |
| alaski | mriedem: yep, it's going to be all over any run that hasn't migrated to cellsv2 | 17:24 |
| alaski | https://review.openstack.org/367510 | 17:24 |
| openstackgerrit | Paul Carlton proposed openstack/nova: Update instance group during live migration https://review.openstack.org/339588 | 17:25 |
| mriedem | could we keep the warning if we can detect cells v2 setup has been done, like if the cell0 mapping exists? | 17:25 |
| alaski | mriedem: I considered that. It would require a db lookup and I wasn't sure it was worth it for a log message. But I'm happy to add it | 17:26 |
| mriedem | yeah...not sure | 17:26 |
| mriedem | alaski: i'd say we do that after we make cellsv2 the mainline default in devstack | 17:28 |
| *** unicell has joined #openstack-nova | 17:28 | |
| alaski | mriedem: put back the warning? | 17:29 |
| *** thorst has joined #openstack-nova | 17:29 | |
| *** rlrossit has joined #openstack-nova | 17:29 | |
| mriedem | yeah | 17:29 |
| mriedem | or do that lookup | 17:29 |
| mriedem | well, warning conditional on the lookup | 17:29 |
| alaski | okay. I'll add a TODO there | 17:29 |
| mriedem | but after we make cellsv2 the default path in devstack | 17:29 |
| mriedem | ok | 17:29 |
| *** tongli has joined #openstack-nova | 17:30 | |
| edleafe | cdent: commented on https://review.openstack.org/#/c/366789 | 17:30 |
| mriedem | gibi: btw, that warning on the notification thing is only showing up in the cellsv1 job | 17:31 |
| cdent | edleafe: the thing we want to achieve is allow comparisons of dicts with minimal manipulation so having generation and resource classes at the same level in the dict is not desirable | 17:32 |
| cdent | i'm late for dinner, or I'd stop and say more | 17:32 |
| openstackgerrit | Sam Betts proposed openstack/nova: [WIP] Support Ironic interface attach/detach in nova virt https://review.openstack.org/364413 | 17:32 |
| burgerk | mriedem: alaski: new patch uploaded https://review.openstack.org/#/c/366955/ | 17:33 |
| openstackgerrit | Andrew Laski proposed openstack/nova: Remove misleading warning message https://review.openstack.org/367510 | 17:33 |
| *** tongli has quit IRC | 17:34 | |
| *** code-R has joined #openstack-nova | 17:35 | |
| *** thorst has quit IRC | 17:37 | |
| *** ssurana has joined #openstack-nova | 17:37 | |
| *** sudipto_ has quit IRC | 17:37 | |
| *** sudipto has quit IRC | 17:37 | |
| *** code-R_ has joined #openstack-nova | 17:38 | |
| *** rlrossit has quit IRC | 17:39 | |
| edleafe | cdent: agree, and really don't understand the need for returning generation | 17:39 |
| *** abalutoiu has joined #openstack-nova | 17:39 | |
| edleafe | cdent: but if it's gotta be there... | 17:40 |
| *** swamireddy has joined #openstack-nova | 17:40 | |
| *** code-R has quit IRC | 17:42 | |
| *** rrecio has quit IRC | 17:44 | |
| *** rama_y has quit IRC | 17:44 | |
| *** ducttape_ has quit IRC | 17:45 | |
| *** whenry has quit IRC | 17:47 | |
| *** tongli has joined #openstack-nova | 17:48 | |
| *** gszasz has quit IRC | 17:49 | |
| *** krtaylor has quit IRC | 17:49 | |
| dansmith | edleafe: right, currently the generation does nothing for us from the client side during allocation activities | 17:50 |
| *** PaulMurray has quit IRC | 17:50 | |
| dansmith | it may in the future if we try to do a more consolidated RT update sorta thing, but right now it's superfluous | 17:50 |
| *** tongli has quit IRC | 17:52 | |
| *** akshai has quit IRC | 17:54 | |
| *** jlk has joined #openstack-nova | 17:54 | |
| cdent | dansmith, edleafe: I"m thinking:$.allocations.$provider_uuid.generation: 5 | 17:54 |
| cdent | $.allocations.$resource_uuid.resources: {VCPU: 4, DISK_GB: 20} | 17:55 |
| dansmith | cdent: yeah that's what I suggested earlier, | 17:55 |
| *** su_zhang has quit IRC | 17:55 | |
| dansmith | so obviously I'm cool with that :D | 17:55 |
| cdent | so effectively what jay has done, but with s/resource_providers/allocations/ | 17:55 |
| cdent | yeah, cool | 17:55 |
| jlk | Hi all. I've got a couple mitaka clouds and we're trying to get NUMA scheduling tested. But first, we seem to not be getting any NUMA details reported into novadb from the compute host. Selecting "numa_topology from compute_nodes" is showing NULL. Could somebody help me debug where this info is getting lost? | 17:55 |
| cdent | it preserves the key part of having a comparable piece at a predictable key | 17:55 |
| jlk | if I understand things, that data should be coming through if the hardware itself has NUMA details | 17:56 |
| edleafe | cdent: that sounds good to me | 17:56 |
| cdent | ✔ | 17:56 |
| dansmith | cdent: yes | 17:56 |
| cdent | coming soon to a gerrit near you | 17:56 |
| *** prateek has joined #openstack-nova | 17:57 | |
| mriedem | jlk: have you gone through http://docs.openstack.org/admin-guide/compute-cpu-topologies.html ? | 17:59 |
| mriedem | jlk: cfriesen, sfinucan or lbeliveau might be able to help | 17:59 |
| jlk | mriedem: We were reading an old version something like that http://docs.openstack.org/developer/nova/testing/libvirt-numa.html | 18:00 |
| jlk | let me read this new link and see if it gells | 18:00 |
| mriedem | jlk: yeah that devref one is pretty old | 18:00 |
| *** fragatina has joined #openstack-nova | 18:01 | |
| jlk | eh | 18:01 |
| jlk | that link you sent doesn't cover anything other than "turn on the filter" | 18:01 |
| jlk | We've got the filter in, and it fails ever attempt to build with NUMA, stating no hosts match | 18:01 |
| jlk | and since the numa data is all NULL, I can see why | 18:01 |
| *** kfarr has quit IRC | 18:02 | |
| *** akshai has joined #openstack-nova | 18:02 | |
| mriedem | i swear there is another doc about configuring for nfv things like numa, and lbeliveau helped work on that | 18:02 |
| mriedem | and sfinucan | 18:02 |
| cfriesen | jlk: does "virsh capabilities" show numa node information? | 18:02 |
| mriedem | alaski: just had a cellsv1 failure with this http://logs.openstack.org/55/367455/1/check/gate-tempest-dsvm-cells/1d8af83/logs/screen-n-api.txt.gz#_2016-09-08_15_59_02_553 got a 500 from the api on delete | 18:03 |
| mriedem | alaski: in this test, it creates the server and then immediately deletes it | 18:03 |
| mriedem | doesn't wait for active | 18:03 |
| *** rlrossit has joined #openstack-nova | 18:04 | |
| jlk | cfriesen: I see this <topology> <cells num='2'> | 18:05 |
| jlk | am I looking for that, or something else? | 18:05 |
| *** vishwanathj is now known as vishwanathj_ | 18:06 | |
| jlk | maybe that's somethin gelse. | 18:06 |
| *** twaldrop has joined #openstack-nova | 18:06 | |
| *** swamireddy has quit IRC | 18:06 | |
| cfriesen | jlk: that's promising...then you should see <cell id='0'> with some "pages" entries and some cpus, then <cell id='1'> and some memory/cpu info | 18:06 |
| jlk | correct | 18:06 |
| *** wasmum has quit IRC | 18:06 | |
| cdent | jaypipes: if I was going to make my decision to visit your home based on your recent tweets: _no way_ | 18:07 |
| jlk | <cpu id='0' socket_id='0' core_id='0' siblings='0,24'/> | 18:07 |
| *** ducttape_ has joined #openstack-nova | 18:07 | |
| cfriesen | jlk: okay, that's good. what version of libvirt/qemu are you using? | 18:08 |
| jaypipes | cdent: ikr? | 18:08 |
| *** swamireddy has joined #openstack-nova | 18:08 | |
| alaski | mriedem: okay. looks like there's a small race there. I can add some exception handling | 18:08 |
| jaypipes | edleafe: my issue with your proposed response format there is that then we need some kind of tribal knowledge to know that "generation" is not a resource class identifier. | 18:08 |
| jlk | cfriesen: libvirt is 1.2.2-0ubuntu13.1.17 qemu-kvm is 2.0.0+dfsg-2ubuntu1.22 | 18:09 |
| cfriesen | jlk: just to confirm, you're using libvirt as the hypervisor? | 18:09 |
| jaypipes | edleafe: and the generation is *not* an attribute of the allocation/resource record, but rather the resource provider itself. | 18:09 |
| jaypipes | edleafe: so it feels, well, weird :) | 18:09 |
| mriedem | alaski: https://bugs.launchpad.net/nova/+bug/1621574 | 18:09 |
| openstack | Launchpad bug 1621574 in OpenStack Compute (nova) "Instance fails to delete while building in cells v1 due to "ObjectActionError: Object action destroy failed because: host changed"" [Undecided,Confirmed] | 18:09 |
| mriedem | alaski: is that related to BuildRequests? | 18:09 |
| sdague | alaski: thanks for https://review.openstack.org/#/c/367510/2 - +A | 18:09 |
| *** ociuhandu has joined #openstack-nova | 18:10 | |
| alaski | mriedem: yes. It's because we don't have this try/except also around line 224 http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/cells_api.py#n242 | 18:10 |
| cfriesen | jlk: ah, here we go. in virt/libvirt/driver.py, MIN_LIBVIRT_NUMA_VERSION = (1, 2, 7) and MIN_LIBVIRT_HUGEPAGE_VERSION = (1, 2, 8) | 18:10 |
| mriedem | alaski: cool | 18:11 |
| cfriesen | jlk: your version of libvirt is too old for NUMA support | 18:11 |
| mriedem | jlk: time to move to xenial | 18:11 |
| mriedem | or, kilo cloud archive | 18:11 |
| jlk | cfriesen: yeah, libvirt as the HV. | 18:12 |
| jlk | oh wtf? | 18:12 |
| *** kaisers has joined #openstack-nova | 18:12 | |
| jlk | We had been using a newer version at some point, I wonder what changed | 18:12 |
| mriedem | larry in accounting happened | 18:13 |
| edleafe | jaypipes: Agreed, but it feels weird to have generation in there in the first place. | 18:13 |
| cfriesen | jlk: just as a heads-up, if you're enabling NUMATopologyFilter you're probably going to want to somehow enable debug logs, at least for that filter. There are a bunch of obscure ways that a host can fail the filter. | 18:14 |
| *** akshai has quit IRC | 18:14 | |
| *** lakshmiS has joined #openstack-nova | 18:14 | |
| jlk | yeah, that's our plan. this is for a potential customer to test a work load, not a thing we'd want to turn on all the time | 18:14 |
| *** ihrachys has quit IRC | 18:15 | |
| *** lakshmiS has left #openstack-nova | 18:15 | |
| jaypipes | edleafe: the alternative is to do multiple REST API calls and hope that something doesn't change in between the two calls, which isn't much of an alternative IMHO :P | 18:16 |
| *** efoley has joined #openstack-nova | 18:17 | |
| *** lpetrut has quit IRC | 18:17 | |
| *** rama_y has joined #openstack-nova | 18:17 | |
| *** kaisers has quit IRC | 18:17 | |
| *** shangxdy has quit IRC | 18:18 | |
| sdague | dansmith: that's an odd objects failure - http://logs.openstack.org/55/367455/1/check/gate-tempest-dsvm-cells/1d8af83/logs/screen-n-api.txt.gz?level=INFO#_2016-09-08_15_59_02_553 - is that just because cells v1 is odd? | 18:20 |
| dansmith | sdague: hmm | 18:21 |
| dansmith | sdague: that's a sanity check, so it's good that it's failing, but not sure when it would happen in the gate | 18:21 |
| mriedem | dansmith: sdague: scroll up | 18:21 |
| dansmith | oh in delete | 18:21 |
| *** nic has left #openstack-nova | 18:21 | |
| *** krtaylor has joined #openstack-nova | 18:24 | |
| *** karthiks has quit IRC | 18:24 | |
| sdague | yeh, it's in the tear down of some servers | 18:24 |
| mriedem | i emailed sahid but i think we have a bug with the pointer_model config option that was added in newton | 18:24 |
| mriedem | http://logs.openstack.org/26/367526/1/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/1479bdd/logs/screen-n-api.txt.gz?level=TRACE#_2016-09-08_17_31_42_490 | 18:24 |
| mriedem | https://review.openstack.org/#/c/367526/ | 18:24 |
| *** lpetrut has joined #openstack-nova | 18:25 | |
| mriedem | so use_usb_tablet is deprecated, but defaults to True. replaced by pointer_model, which defaults to None. | 18:25 |
| *** tongli has joined #openstack-nova | 18:25 | |
| mriedem | if you don't set pointer_model, we fallback to using use_usb_tablet | 18:25 |
| mriedem | but the defaults have different meanings | 18:25 |
| mriedem | and if you want that thing off, you can't really straddle it | 18:25 |
| jlk | so uh, related questions. | 18:26 |
| jlk | what concerns are there when upgrading qemu and libvirt versions on an existing cloud? | 18:26 |
| mriedem | upgrading libvirt shouldn't be an issue since it's an api over qemu | 18:27 |
| mriedem | restarting qemu would restart the guests | 18:27 |
| mriedem | so i'd think you'd need to hot patch qemu or migrate instances off that compute prior to the upgrade | 18:27 |
| *** sdake_ is now known as sdake | 18:27 | |
| sdague | qemu doesn't restart per say | 18:27 |
| jlk | right it's just a series of running processes | 18:28 |
| sdague | the guests will just be on the old version until you bounce them | 18:28 |
| *** tongli has quit IRC | 18:28 | |
| jlk | that's fine | 18:28 |
| *** tongli has joined #openstack-nova | 18:28 | |
| sdague | I believe you can just do the upgrade in place, then nova reboot on the computes when you can handle an outage for each compute, and you'll be fine | 18:29 |
| dansmith | yep | 18:29 |
| jlk | and what if they just are left running until the customer does a reboot action? | 18:29 |
| jlk | or rebuild or whatever? | 18:29 |
| dansmith | jlk: yep, fine | 18:30 |
| sdague | then they are left running | 18:30 |
| jlk | alright. I thought at one point there were some issues with the XML formats | 18:30 |
| dansmith | nope | 18:30 |
| jlk | and some scenarios where things would go sideways | 18:30 |
| dansmith | not that I know of | 18:30 |
| jlk | neat. | 18:30 |
| *** tongli has quit IRC | 18:30 | |
| dansmith | there were some live migration issues because of a bug in qemu | 18:30 |
| sdague | while things are running, it's fine. They aren't getting new xml fed to them anyway, it's just a process | 18:30 |
| dansmith | but nothing I've ever heard of regarding them being on a single host | 18:30 |
| *** tongli has joined #openstack-nova | 18:31 | |
| *** RuiChen has quit IRC | 18:31 | |
| *** tongli has quit IRC | 18:32 | |
| *** tongli has joined #openstack-nova | 18:32 | |
| *** su_zhang has joined #openstack-nova | 18:33 | |
| sdague | dansmith: hmmm... http://logs.openstack.org/89/366789/2/check/gate-tempest-dsvm-cells/04607f2/logs/screen-n-api.txt.gz?level=ERROR#_2016-09-08_16_40_57_877 | 18:33 |
| sdague | same error on delete in cellsv1 | 18:33 |
| sdague | different test | 18:33 |
| *** tongli has quit IRC | 18:33 | |
| dansmith | sdague: I'm thinking that's an early delete | 18:33 |
| dansmith | sdague: so it gets created with no host and then assigned then deleted? | 18:33 |
| *** paul-carlton has quit IRC | 18:33 | |
| dansmith | I have to run off now, but blame alaski | 18:34 |
| dansmith | it | 18:34 |
| *** tongli has joined #openstack-nova | 18:34 | |
| dansmith | it's not an objects thing regardless.. the assertion is just in that code | 18:34 |
| *** thorst has joined #openstack-nova | 18:34 | |
| alaski | I need to replicate the try except at http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/cells_api.py#n242 a few lines up on line 224 | 18:35 |
| sdague | ok, it just failed one of the placement patches on the hot list | 18:35 |
| alaski | I have a patch, just figuring out testing | 18:35 |
| sdague | alaski: ok, great | 18:35 |
| *** akshai has joined #openstack-nova | 18:35 | |
| mriedem | sdague: dansmith: yes it's an early delete | 18:35 |
| mriedem | the tests are deleting the server while it's building | 18:35 |
| mriedem | sdague: i pushed the e-r query for this about 20 min ago | 18:36 |
| mriedem | just fyi | 18:36 |
| mriedem | i could use some feedback on this https://bugs.launchpad.net/nova/+bug/1621582 | 18:36 |
| openstack | Launchpad bug 1621582 in OpenStack Compute (nova) "use_usb_tablet and pointer_model have different defaults making switching hard" [Undecided,Confirmed] | 18:36 |
| mriedem | if it's an rc blocker or not | 18:36 |
| *** jichen has joined #openstack-nova | 18:39 | |
| sdague | mriedem: yeh, that seems like an annoying upgrade issue we should avoid | 18:39 |
| *** Swami has joined #openstack-nova | 18:39 | |
| mriedem | i'm annoyed, yes | 18:39 |
| mriedem | :) | 18:39 |
| jlk | Thanks for the help folks, we'll keep plugging away at this with a newer libvirt. | 18:40 |
| *** mfedosin has quit IRC | 18:41 | |
| *** Jeffrey4l__ has quit IRC | 18:42 | |
| *** thorst has quit IRC | 18:42 | |
| mriedem | sdague: added to the newton rc list so we don't forget about it, i've also emailed it to sahid to see if he has ideasd | 18:43 |
| mriedem | it was his bp | 18:43 |
| *** mgoddard has quit IRC | 18:43 | |
| jaypipes | grr, still working on one remaining test failure... | 18:46 |
| *** su_zhang has quit IRC | 18:47 | |
| *** Sukhdev has joined #openstack-nova | 18:51 | |
| *** david-lyle has quit IRC | 18:53 | |
| openstackgerrit | Vladik Romanovsky proposed openstack/nova: libvirt: inject files when config drive is not requested https://review.openstack.org/335676 | 18:53 |
| openstackgerrit | Vladik Romanovsky proposed openstack/nova: cleanup: sparating the creation of a local root to it's own method https://review.openstack.org/367571 | 18:53 |
| *** david-lyle has joined #openstack-nova | 18:53 | |
| *** Jeffrey4l__ has joined #openstack-nova | 18:54 | |
| openstackgerrit | Doug Hellmann proposed openstack/nova: standardize release note page ordering https://review.openstack.org/367578 | 18:56 |
| *** vishwanathj_ is now known as vishwanathj | 19:00 | |
| *** abalutoiu has quit IRC | 19:01 | |
| *** tbachman has joined #openstack-nova | 19:01 | |
| openstackgerrit | Chris Dent proposed openstack/nova: [placement] add two ways to GET allocations https://review.openstack.org/366789 | 19:03 |
| *** rlrossit has quit IRC | 19:04 | |
| cdent | jaypipes, edleafe, dansmith, bauzas ^ I think that hits the right notes on the representations. will add it to the etherpad | 19:04 |
| *** TravT has quit IRC | 19:05 | |
| *** salv-orlando has quit IRC | 19:06 | |
| *** rlrossit has joined #openstack-nova | 19:07 | |
| *** rrecio has joined #openstack-nova | 19:08 | |
| openstackgerrit | Vladik Romanovsky proposed openstack/nova: libvirt: inject files when config drive is not requested https://review.openstack.org/335676 | 19:10 |
| openstackgerrit | Vladik Romanovsky proposed openstack/nova: cleanup: separate the creation of a local root to it's own method https://review.openstack.org/367571 | 19:10 |
| sdague | cdent: out of curiosity, when are we going to start the api-ref doc for all of this? | 19:11 |
| cdent | sdague: an excellent question. a) we probably need to have a more in depth discussion on how we'd like that to be b) we need to wait for us to stop changing things :( | 19:13 |
| cdent | I'm hoping it won't be just me | 19:13 |
| sdague | pseudo related question, are the uuids in the paths format checked? | 19:15 |
| jaypipes | cdent: ++ nice | 19:15 |
| cdent | sdague: you mean is /resource_providers/{uuid} check to be a uuid before we try to load that resource provider? | 19:15 |
| cdent | if that's the question the answer is no | 19:16 |
| cdent | I had some pattern matching on the Routes side that would not match the route if it was not a uuid but then got rid of it so that we had the option to make a better NotFound inside the handlers | 19:17 |
| sdague | ok, so if we send utf8 dolphin down that pipe, what happens? | 19:17 |
| cdent | it should be a 404, I've started some tests near that vein, but not quite exactly in: | 19:18 |
| cdent | https://review.openstack.org/#/c/365688/ | 19:18 |
| cdent | sdague: btw, I still think we should do the "better noffound inside the handlers" at some point. My understanding of how the review process would flow this cycle was full of bad understandings. | 19:20 |
| sdague | ok, yeh, that doesn't seem to cover this case | 19:20 |
| cdent | yes, exactly, thus "near that vein" | 19:20 |
| cdent | I'm trying it now | 19:20 |
| sdague | ok, cool | 19:20 |
| sdague | for floating ips we are actually returning 400 on badly formatted path element | 19:20 |
| sdague | because it turns out people do silly typos | 19:21 |
| cdent | that seems odd | 19:21 |
| sdague | and just doing 404 to people because they missed an octet in a copy to the command line ends up masking a lot of issues | 19:21 |
| sdague | it exposed a bunch of client bugs where they were trying to do deletes with completely the wrong representation | 19:22 |
| *** coolsvap has quit IRC | 19:22 | |
| edleafe | cdent: got one bug in https://review.openstack.org/#/c/366789, otherwise I'm +1 | 19:23 |
| cdent | sdague I understand the motivation, but if the URI is correctly formed (uses characters that are okay) but there's no resource, that's a 404 | 19:23 |
| edleafe | cdent: that method is a bit too clever | 19:23 |
| cdent | thanks edleafe will look in a sec | 19:23 |
| cdent | edleafe: yeah, I had two different ones before and felt bad about so made one and then felt bad about that too | 19:23 |
| oomichi | sdague: cdent: hi, a question about https://review.openstack.org/#/c/337380 | 19:23 |
| oomichi | sdague: cdent: If flavor setting is invalid and "create a server" API needs to return an error response, what is the best status code? | 19:24 |
| cdent | and then decided to just see what people said | 19:24 |
| sdague | cdent: but if we know that it's not even possible for that uri to exist, it seems weird to not be helpful and tell the user | 19:24 |
| sdague | oomichi: the 400 seems right | 19:24 |
| cdent | sdague: sure, but that would be in the 404's message, not changing the 404 | 19:24 |
| oomichi | sdague: cdent: current patch returns 400, but wrong flavor setting is due to operator, not user. 400 doesn't seem right | 19:24 |
| *** moshele has joined #openstack-nova | 19:25 | |
| oomichi | sdague: cdent: or should we consider users specify wrong flavor? | 19:25 |
| cdent | oomichi: i haven't had a chance to read it yet, what's the alternative to 400? | 19:25 |
| sdague | cdent: well, turns out people assume 404s are because they already deleted the thing in question | 19:26 |
| mriedem | hemanthm: in addition to those examples, the biggest online data migration, and i think the first, was moving flavor info from instance.system_metadata to instance.flavor in kilo | 19:26 |
| cdent | sdague: I don't know how to respond to that. Some people assume stop signs are optional and suffer no consequences as a result, but doing so isn't right. | 19:28 |
| mriedem | hemanthm: this will probably hurt your eyes, but https://review.openstack.org/#/c/135700/ | 19:28 |
| mriedem | hemanthm: like i said, that was the biggest and most complicated online data migration | 19:28 |
| cdent | (snowman resource provider uuid gives a 404 as expected, will add those tests) | 19:28 |
| mriedem | hemanthm: with the nova db to api db migrations, those are mostly a pattern of new things go to the api db, we do lookups in the api db first and then fallback to the nova (old) db, and then have a nova-manage command to force that migration during upgrades | 19:29 |
| mriedem | we don't migrate on lookup like in https://review.openstack.org/#/c/135700/ | 19:29 |
| mriedem | hemanthm: i'm assuming glance will have a summit session about rolling upgrade plans? | 19:30 |
| mriedem | and keystone probably is too? | 19:30 |
| stevemar | mriedem: probably | 19:30 |
| hemanthm | mriedem: yep, there's a fishbown session on rolling upgrades | 19:30 |
| hemanthm | *fishbowl | 19:30 |
| mriedem | hemanthm: you can probably do similar things w/o versioned objects for these data migrations | 19:31 |
| mriedem | we do them in the objects since that's where we access the db apis and it's a convenient abstraction | 19:31 |
| stevemar | mriedem: depends on who the sucker^H^H^H ptl is at the time | 19:31 |
| hemanthm | mriedem: +1, that's what I'm trying to research. | 19:32 |
| mriedem | hemanthm: but like in this series https://review.openstack.org/#/q/topic:bp/cells-keypairs-api-db i don't think we actually needed to bump the version on the instance object, except when adding the new field | 19:33 |
| mriedem | otherwise the data migrations don't require a version bump | 19:33 |
| hemanthm | I guess putting all that logic into db layer will probably make it a lot more messy | 19:33 |
| *** su_zhang has joined #openstack-nova | 19:33 | |
| openstackgerrit | Kenneth Burger proposed openstack/nova: Increase BDM column in build_requests table https://review.openstack.org/366955 | 19:33 |
| hemanthm | and oslo.vo maybe an overkill for glance at this point | 19:33 |
| mriedem | well, you're going to have conditionals somewhere | 19:33 |
| mriedem | stevemar: that's not going to be you? | 19:34 |
| *** mgoddard has joined #openstack-nova | 19:36 | |
| stevemar | mriedem: i can be a sucker too | 19:38 |
| *** su_zhang has quit IRC | 19:38 | |
| *** thorst has joined #openstack-nova | 19:39 | |
| openstackgerrit | Chris Dent proposed openstack/nova: [placement] Add some tests ensuring unicode resource provider info https://review.openstack.org/365688 | 19:39 |
| *** cmartin_ has quit IRC | 19:40 | |
| oomichi | cdent: yeah, that is my question. In this case, operator needs to fix the flavor. I cannot find better one, tbh | 19:40 |
| cdent | oomichi: I don't think you want anything in the 5xx range, and none of the specific options in the 4xx range seem to fix, so 400 is the reasonable fallback | 19:40 |
| oomichi | cdent: ok, now I feel 400 can cover this case also like https://github.com/openstack/api-wg/blob/master/guidelines/http.rst#use-of-501---not-implemented | 19:40 |
| *** efoley has quit IRC | 19:40 | |
| oomichi | cdent: yeah, I cannot find valid 50x in this case | 19:41 |
| * cdent nods | 19:41 | |
| *** Apoorva has quit IRC | 19:41 | |
| oomichi | cdent: it seems enough to report back a reson in response. | 19:41 |
| oomichi | cdent: sdague: I got it, thanks :) | 19:42 |
| *** tongli_ has joined #openstack-nova | 19:43 | |
| *** burgerk has quit IRC | 19:43 | |
| openstackgerrit | jichenjc proposed openstack/nova: Catch DevicePathInUse in attach_volume https://review.openstack.org/367358 | 19:44 |
| openstackgerrit | Andrew Laski proposed openstack/nova: Handle ObjectActionError during cells instance delete https://review.openstack.org/367597 | 19:44 |
| * cdent is the king of adding undesirable whitespace | 19:45 | |
| *** tongli has quit IRC | 19:46 | |
| *** thorst has quit IRC | 19:47 | |
| *** code-R_ has quit IRC | 19:47 | |
| *** yamahata has joined #openstack-nova | 19:49 | |
| *** sdake_ has joined #openstack-nova | 19:50 | |
| *** Jeffrey4l__ has quit IRC | 19:50 | |
| openstackgerrit | Chris Dent proposed openstack/nova: [placement] add two ways to GET allocations https://review.openstack.org/366789 | 19:50 |
| melwitt | I triaged a numa bug recently where it looks like we've been using the wrong image metadata key names for a few things. is that something we would want to look at as rc potential? https://bugs.launchpad.net/nova/+bug/1568191 | 19:50 |
| openstack | Launchpad bug 1568191 in Glance "Pre-defined VirtCPUTopology metadata should have the same name as flavor extra specs in nova" [Undecided,In progress] - Assigned to Ying Zuo (yingzuo) | 19:50 |
| openstackgerrit | jichenjc proposed openstack/nova: refactory query user_id code for quota-sets https://review.openstack.org/280165 | 19:51 |
| melwitt | that is assuming that glance is the source of truth for the key names | 19:51 |
| *** sdake has quit IRC | 19:52 | |
| mriedem | melwitt: sfinucan has a comment in there, "For things like CPU policies, image metadata will take precedence over flavor extra specs, but both are valid." | 19:53 |
| mriedem | that's how i've seen things work for some other keys, there are image meta keys and flavor extra specs, and not always the same | 19:53 |
| mriedem | but one takes precedence | 19:53 |
| melwitt | mriedem: yeah, but if we pull the data using the wrong key names we won't get it? | 19:53 |
| * cdent waves | 19:54 | |
| *** cdent has quit IRC | 19:54 | |
| mriedem | melwitt: the code would need to know to use the proper key name right | 19:54 |
| mriedem | and why they'd be different idk | 19:54 |
| mriedem | that's just asking for problems | 19:54 |
| *** prateek has quit IRC | 19:54 | |
| sdague | mriedem: because they namespace differently I think | 19:54 |
| sdague | https://github.com/openstack/glance/blob/1c242032fbb26fed3a82691abb030583b4f8940b/etc/metadefs/compute-vcputopology.json#L19 vs https://github.com/openstack/glance/blob/1c242032fbb26fed3a82691abb030583b4f8940b/etc/metadefs/compute-vcputopology.json#L10 | 19:55 |
| *** ducttape_ has quit IRC | 19:55 | |
| sdague | oh, there is an extra _ in the nova names? | 19:56 |
| mriedem | yeah there was something for a bp in newton that i was reviewing that was like this, i think the hyper-v uefi stuff | 19:56 |
| melwitt | ours is for flavor extra specs, for example hw:cpu_max_sockets and in glance image meta it's hw_cpu_maxsockets | 19:56 |
| *** kaisers has joined #openstack-nova | 19:56 | |
| sdague | https://github.com/openstack/nova/blob/f5c0b59fd68e88aa37ec69b8e544c31e6809da44/nova/virt/hardware.py#L273-L280 | 19:56 |
| *** Oku_OS is now known as Oku_OS-away | 19:56 | |
| mriedem | https://review.openstack.org/#/c/209581/45/releasenotes/notes/hyperv-uefi-secure-boot-a2a617ac2c313afd.yaml | 19:57 |
| sdague | melwitt: so, mostly it's fine that they are different, but it looks like our definitions for glance are just wrong | 19:57 |
| melwitt | the person proposed a change to change glance's key names to match ours, it's currently -2ed https://review.openstack.org/#/c/303658/ | 19:57 |
| melwitt | sdague: right. that's what it appeared to me too | 19:57 |
| sdague | melwitt: so you just need to delete some underscores right? | 19:57 |
| sdague | https://github.com/openstack/nova/blob/f5c0b59fd68e88aa37ec69b8e544c31e6809da44/nova/virt/hardware.py#L316-L318 | 19:58 |
| melwitt | sdague: I think so | 19:58 |
| mriedem | for that hyperv uefi one the pattern is the same, image meta starts with os_* and extra spec starts with os:* | 19:58 |
| sdague | mriedem: right, I don't think that kind of change is a thing here, but the narrower one that melwitt has is probably fine for rc | 19:58 |
| sdague | it should be a pretty small patch | 19:58 |
| mriedem | is this a regression? | 19:59 |
| melwitt | I don't think it's a regression. afaict it's been like this | 19:59 |
| mriedem | yeah, so not an rc blocker, but simple sounding fix | 19:59 |
| mriedem | and we could backport it | 19:59 |
| sdague | mriedem: we could, but it also doesn't get in the way to get it in | 20:00 |
| sdague | we wouldn't hold the rc for the bug | 20:00 |
| *** kaisers has quit IRC | 20:00 | |
| mriedem | agree | 20:00 |
| sdague | but it's a valid bug I think to fix | 20:00 |
| sdague | before the rc | 20:00 |
| melwitt | does the newton-rc-potential tag mean "hold the release for it"? | 20:01 |
| sdague | melwitt: yes | 20:01 |
| mriedem | melwitt: yup | 20:01 |
| melwitt | okay, thanks | 20:01 |
| mriedem | it at least means, potentially hold the release | 20:01 |
| sdague | so don't tag it that way, but if you get a patch up with it and it gets in, we're good. This seems super quick to get in. | 20:01 |
| mriedem | melwitt: yeah just ping for attention | 20:02 |
| mriedem | i know you're usually very pushy about people reviewing your code and all :) | 20:02 |
| sdague | heh | 20:02 |
| mriedem | you also suffer from laski syndrome | 20:02 |
| melwitt | okay, cool. I haven't noticed a trend in the tagged ones i.e. they're not all regressions so I wasn't sure | 20:02 |
| melwitt | haha, that's true :) | 20:03 |
| *** moshele has quit IRC | 20:03 | |
| mtreinish | mriedem: that sounds contagious | 20:05 |
| mriedem | mtreinish: i don't think so, | 20:05 |
| mriedem | people don't generally get nicer | 20:06 |
| mriedem | bauzas: is this a regression in newton? https://bugs.launchpad.net/nova/+bug/1600251 | 20:06 |
| openstack | Launchpad bug 1600251 in OpenStack Compute (nova) "live migration does not honor server group policy" [High,In progress] - Assigned to Paul Carlton (paul-carlton2) | 20:06 |
| mriedem | since it's marked for mitaka-backport-potential i'm assuming not | 20:06 |
| mriedem | paul-carlton2: ^ | 20:07 |
| bauzas | mriedem: that regressed in Mitaka unfortunatly :( | 20:07 |
| *** cdelatte has quit IRC | 20:08 | |
| oomichi | mriedem: could you go back to https://review.openstack.org/#/c/358152/ again? | 20:10 |
| *** su_zhang has joined #openstack-nova | 20:10 | |
| mriedem | oomichi: i'm not sure what "So I am not sure we need to drop this on Nova side or not." means | 20:11 |
| oomichi | mriedem: ah, sorry about that. I wanted to mean the corresponding neutron patch is merged into liberty branch as https://review.openstack.org/#/c/338200/ | 20:13 |
| mriedem | oomichi: i'm not sure how the neutron patch is related | 20:14 |
| mriedem | oomichi: does the neutron change mean that the nova error won't happen? | 20:14 |
| mriedem | if you have that neutron change i mean | 20:14 |
| mriedem | i.e. nova won't get the port not found that the nova change is handling | 20:14 |
| mriedem | in other words, will fip['port_id'] in https://review.openstack.org/#/c/358152/1/nova/network/neutronv2/api.py just be None? | 20:15 |
| *** Guest60381 has quit IRC | 20:16 | |
| *** praveen has quit IRC | 20:16 | |
| *** mgoddard has quit IRC | 20:16 | |
| *** Oku_OS-away is now known as Oku_OS | 20:16 | |
| mriedem | oomichi: the other thing was there were no logstash hits for that failure, which to me means we're not hitting this often in the gate | 20:17 |
| mriedem | so i don't see the justification for landing that in liberty if it's not really going to benefit tempest runs | 20:18 |
| *** su_zhang has quit IRC | 20:18 | |
| *** salv-orlando has joined #openstack-nova | 20:18 | |
| *** ducttape_ has joined #openstack-nova | 20:18 | |
| oomichi | mriedem: ok, I will check the code deeply later. About the policy of stable branches, do backporting patches need to be verified on the gate? | 20:20 |
| *** yamahata has quit IRC | 20:20 | |
| mriedem | oomichi: well we run ci on them | 20:21 |
| mriedem | oomichi: but i'm not sure that's what you're asking | 20:21 |
| mriedem | oomichi: my point is, stable/liberty is more or less locked down | 20:21 |
| mriedem | http://docs.openstack.org/project-team-guide/stable-branches.html | 20:22 |
| mriedem | liberty is in phase 3 | 20:22 |
| mriedem | so we need justification for landing things in liberty, like if they are fixing a security or critical bug, or if landing them would unblock something else, like a new tempest test / change for example, or unblock the gate | 20:22 |
| *** salv-orlando has quit IRC | 20:22 | |
| mriedem | from a high percentage test failure | 20:22 |
| oomichi | mriedem: ah, I saw the point. (Phase III (more than 12 months): Only security patches are acceptable) | 20:23 |
| *** su_zhang has joined #openstack-nova | 20:23 | |
| mriedem | it's not always security fixes, | 20:23 |
| mriedem | but phase 3 branch changes get more scrutiny | 20:23 |
| oomichi | mriedem: I got it, thanks for the explanation | 20:24 |
| oomichi | mriedem: I just had a question about difference between mitaka patch and liberty patch, that is clear now | 20:25 |
| *** Oku_OS is now known as Oku_OS-away | 20:26 | |
| mriedem | np | 20:27 |
| *** jichen has quit IRC | 20:30 | |
| *** Oku_OS-away is now known as Oku_OS | 20:31 | |
| *** TravT has joined #openstack-nova | 20:31 | |
| *** su_zhang has quit IRC | 20:32 | |
| *** salv-orlando has joined #openstack-nova | 20:33 | |
| *** thorst has joined #openstack-nova | 20:34 | |
| mriedem | alaski: what was the donut place in portland that you loved? | 20:37 |
| alaski | blue star | 20:37 |
| mriedem | dansmith: i don't want to alarm you, but my parents are going to be in portland next week | 20:37 |
| mriedem | i've informed them that you live in a mountain stronghold on TMNT ave | 20:37 |
| dansmith | mriedem: do I have to see them? | 20:37 |
| dansmith | haha | 20:37 |
| sdague | dansmith: you back and about? In looking at the request timings with / without my logging patch - https://review.openstack.org/#/c/366186/ - it's really only on order a couple of ms per request. Which, during our initial phase here isn't a huge cost for being able to know if things are working | 20:38 |
| *** hamzy has quit IRC | 20:38 | |
| sdague | and we can tune it down later once our comfort level goes up | 20:38 |
| dansmith | sdague: yeah, I hear you, I just don't think we should do that | 20:38 |
| sdague | ok, so without that, I really can't debug anything here. | 20:39 |
| dansmith | sdague: I didn't -1 and poked jaypipes and he said he'd think about it | 20:39 |
| dansmith | I dunno where he went though | 20:39 |
| mriedem | btw, there could be a CONF.trace option in ocata https://review.openstack.org/#/c/366422/ | 20:40 |
| mriedem | to hide that behind | 20:40 |
| *** Oku_OS is now known as Oku_OS-away | 20:40 | |
| *** Apoorva has joined #openstack-nova | 20:41 | |
| sdague | mriedem: right, and I'm fine tuning that down later. The cost for this verification seems really low | 20:42 |
| dansmith | you're doing a bunch of other DB queries every time we do something that was supposed to be a single transaction into the database | 20:42 |
| *** thorst has quit IRC | 20:42 | |
| sdague | they are all reads, and they are cheap reads based on the timing there | 20:43 |
| dansmith | the measured penalty in a gate scenario with a few allocations per provider doesn't really mean much to me | 20:43 |
| sdague | ok, that's fine, I'm totally ok not doing anything more on the placement api side. But I couldn't figure out any of the allocation issues without this patch. | 20:44 |
| *** burgerk has joined #openstack-nova | 20:44 | |
| dansmith | is that like "I refuse to do anything else if we don't merge my logging patch" ? :) | 20:45 |
| sdague | this is the way that I can see that things actually work, without it, I feel like we're pretty blind | 20:45 |
| *** ducttape_ has quit IRC | 20:46 | |
| sdague | anyway, it's dinner time, catch you all tomorrow | 20:46 |
| dansmith | but see, | 20:46 |
| dansmith | you can get all that data from the /usage part of the api | 20:46 |
| *** Apoorva has quit IRC | 20:46 | |
| dansmith | so logging the allocations and the inventories is definitely cool, | 20:47 |
| dansmith | but we can poke the usage calculation externally any time we need, which seems like the appropriate thing to do here | 20:47 |
| dansmith | but again, just MHO, I didn't even -1 | 20:47 |
| *** russellb has left #openstack-nova | 20:48 | |
| dansmith | mriedem: I'm cool with that trace patch... should I review/+2? | 20:49 |
| dansmith | mriedem: and your reno says "may bring system to knees" so seems more appropriate to hide something that does extra queries behind too | 20:50 |
| openstackgerrit | melanie witt proposed openstack/nova: Correct image metadata key names to glance definition https://review.openstack.org/367627 | 20:51 |
| *** sdague has quit IRC | 20:52 | |
| *** tongli_ has quit IRC | 20:52 | |
| *** tongli has joined #openstack-nova | 20:53 | |
| *** ducttape_ has joined #openstack-nova | 20:55 | |
| *** tongli_ has joined #openstack-nova | 20:56 | |
| *** Guest83073 has quit IRC | 20:57 | |
| *** tongli has quit IRC | 20:57 | |
| *** mvk has joined #openstack-nova | 20:58 | |
| *** harlowja_ has joined #openstack-nova | 20:59 | |
| *** harlowja has quit IRC | 21:00 | |
| *** abalutoiu has joined #openstack-nova | 21:01 | |
| mriedem | dansmith: are you oslo.log core? | 21:02 |
| *** pwnall1 has joined #openstack-nova | 21:02 | |
| *** su_zhang has joined #openstack-nova | 21:03 | |
| *** abalutoiu has quit IRC | 21:03 | |
| *** pwnall1 has quit IRC | 21:07 | |
| *** pwnall1 has joined #openstack-nova | 21:08 | |
| *** su_zhang has quit IRC | 21:08 | |
| *** mmedvede has quit IRC | 21:09 | |
| *** devananda is now known as devananda|bbl | 21:09 | |
| *** pwnall1 has quit IRC | 21:09 | |
| mriedem | can we do a thing where we call the usage API only if we hit a conflict or failure in the RT as a debug tool? | 21:11 |
| *** kevinz has joined #openstack-nova | 21:12 | |
| rlrossit | I'll jump in here because I apparently care about logging now too... with sdague's patch, even if we have log.trace, it's still going to do all of those operations right? and then the only configurable part is logging it or not | 21:13 |
| rlrossit | is there any case of if we're doing logging, do all of these heavy operations and log it out, else noop? | 21:14 |
| burgerk | mriedem: do you know of any examples of checking if ia column is MEDIUMTEXT. It seems the column is just returning TEXT for the type field | 21:16 |
| mriedem | burgerk: not sure, would have to look for existing migrations that change a type on a field | 21:16 |
| mriedem | burgerk: zzzeek might know | 21:16 |
| *** nherciu has quit IRC | 21:17 | |
| *** kevinz has quit IRC | 21:17 | |
| alaski | mriedem: https://review.openstack.org/#/c/367597/ | 21:18 |
| *** ametts has quit IRC | 21:19 | |
| mriedem | yup had it open but then got distracted | 21:20 |
| alaski | cool. I just read scrollback and saw your comment to melwitt so figured I had better ping you | 21:21 |
| melwitt | heh | 21:22 |
| *** pwnall1 has joined #openstack-nova | 21:23 | |
| *** tongli_ has quit IRC | 21:26 | |
| *** tonytan4ever has quit IRC | 21:27 | |
| *** cleong has quit IRC | 21:30 | |
| *** pwnall1 has quit IRC | 21:30 | |
| *** ametts has joined #openstack-nova | 21:31 | |
| mriedem | melwitt: how does your image meta change not break on a version bump? | 21:34 |
| mriedem | since the field is being renamed? | 21:34 |
| melwitt | mriedem: I considered that but I thought since the keys are wrong it's currently a no-op? I'm not sure | 21:36 |
| mriedem | melwitt: probably need to ask dansmith | 21:37 |
| mriedem | seems like an upgrade issue though | 21:37 |
| mriedem | like, i'd think we'd need 3 new keys that translate for the old keys if used | 21:37 |
| mriedem | and then we drop the old keys in v2.0 of that object | 21:37 |
| melwitt | hm | 21:38 |
| *** thorst has joined #openstack-nova | 21:39 | |
| *** rmcall has joined #openstack-nova | 21:40 | |
| *** lpetrut has quit IRC | 21:41 | |
| melwitt | mriedem: well, the patch just failed the "test object versions" test so looks like you're right | 21:42 |
| mriedem | YES! | 21:42 |
| melwitt | :) | 21:42 |
| *** diana_clarke has joined #openstack-nova | 21:43 | |
| *** whenry has joined #openstack-nova | 21:44 | |
| *** kaisers has joined #openstack-nova | 21:45 | |
| *** thorst has quit IRC | 21:47 | |
| *** kaisers has quit IRC | 21:49 | |
| *** whenry has quit IRC | 21:50 | |
| *** mmedvede has joined #openstack-nova | 21:53 | |
| *** su_zhang has joined #openstack-nova | 21:53 | |
| *** tblakes has quit IRC | 21:54 | |
| *** ametts has quit IRC | 21:57 | |
| *** annegentle has quit IRC | 21:58 | |
| *** annegentle has joined #openstack-nova | 21:59 | |
| mriedem | alaski: question inline in https://review.openstack.org/#/c/367597/ | 22:01 |
| *** rlrossit has quit IRC | 22:02 | |
| *** rtheis has quit IRC | 22:04 | |
| *** marst has quit IRC | 22:05 | |
| *** jamesdenton has quit IRC | 22:06 | |
| *** burt has quit IRC | 22:09 | |
| *** dimtruck is now known as zz_dimtruck | 22:11 | |
| *** mdrabe has quit IRC | 22:12 | |
| *** tristanC has quit IRC | 22:12 | |
| *** baoli has quit IRC | 22:13 | |
| *** tristanC has joined #openstack-nova | 22:13 | |
| *** baoli has joined #openstack-nova | 22:13 | |
| *** hemna is now known as hemnafk | 22:16 | |
| *** BjoernT has quit IRC | 22:17 | |
| *** harlowja has joined #openstack-nova | 22:18 | |
| *** harlowja_ has quit IRC | 22:22 | |
| *** mvk has quit IRC | 22:23 | |
| *** artom has quit IRC | 22:25 | |
| alaski | mriedem: answered | 22:27 |
| *** tonytan4ever has joined #openstack-nova | 22:28 | |
| *** su_zhang has quit IRC | 22:31 | |
| mriedem | alaski: so if host was set, then cell_name should also be set? | 22:31 |
| mriedem | and elif instance.cell_name should pass? | 22:31 |
| *** claudiub|2 has quit IRC | 22:32 | |
| *** artom has joined #openstack-nova | 22:32 | |
| *** tonytan4ever has quit IRC | 22:33 | |
| *** sdake has joined #openstack-nova | 22:34 | |
| *** sdake_ has quit IRC | 22:36 | |
| *** mriedem has quit IRC | 22:36 | |
| *** mvk has joined #openstack-nova | 22:37 | |
| *** kylek3h has quit IRC | 22:38 | |
| *** akshai has quit IRC | 22:42 | |
| *** BjoernT has joined #openstack-nova | 22:42 | |
| *** thorst has joined #openstack-nova | 22:44 | |
| *** sai has quit IRC | 22:49 | |
| *** krtaylor has quit IRC | 22:51 | |
| *** thorst has quit IRC | 22:53 | |
| *** sai has joined #openstack-nova | 22:54 | |
| *** xyang1 has quit IRC | 22:56 | |
| *** rrecio has quit IRC | 22:59 | |
| *** sdake has quit IRC | 22:59 | |
| *** Sukhdev has quit IRC | 23:00 | |
| *** BjoernT has quit IRC | 23:01 | |
| *** baoli has quit IRC | 23:06 | |
| *** annegentle has quit IRC | 23:08 | |
| *** pradk has quit IRC | 23:08 | |
| *** sai has quit IRC | 23:11 | |
| *** lucas-dinner has quit IRC | 23:12 | |
| *** Apoorva has joined #openstack-nova | 23:18 | |
| *** tjones has joined #openstack-nova | 23:21 | |
| *** kylek3h has joined #openstack-nova | 23:23 | |
| *** markvoelker has quit IRC | 23:25 | |
| *** mriedem has joined #openstack-nova | 23:25 | |
| *** sai has joined #openstack-nova | 23:26 | |
| *** lucasagomes has joined #openstack-nova | 23:26 | |
| *** vladikr has quit IRC | 23:29 | |
| *** jamesdenton has joined #openstack-nova | 23:31 | |
| *** Guest52068 has quit IRC | 23:31 | |
| *** kaisers has joined #openstack-nova | 23:34 | |
| *** annegentle has joined #openstack-nova | 23:36 | |
| *** kaisers has quit IRC | 23:38 | |
| *** annegentle has quit IRC | 23:39 | |
| openstackgerrit | melanie witt proposed openstack/nova: Correct image metadata key names to glance definition https://review.openstack.org/367627 | 23:40 |
| *** gongysh has joined #openstack-nova | 23:40 | |
| *** krtaylor has joined #openstack-nova | 23:45 | |
| mriedem | burgerk: i'm guessing the test was having issues with MediumText because it's not actually a thing in sqla | 23:47 |
| mriedem | it's some variant thing | 23:48 |
| *** devananda|bbl is now known as devananda | 23:48 | |
| *** thorst has joined #openstack-nova | 23:50 | |
| *** Swami has quit IRC | 23:51 | |
| openstackgerrit | melanie witt proposed openstack/nova: Pass GENERATE_HASHES to the tox test environment https://review.openstack.org/367695 | 23:52 |
| *** BjoernT has joined #openstack-nova | 23:52 | |
| mriedem | burgerk: the tests are failing for postgres and sqlite, not mysql | 23:54 |
| mriedem | http://logs.openstack.org/55/366955/3/check/gate-nova-tox-db-functional-ubuntu-xenial/22ae96a/console.html#_2016-09-08_19_51_07_122447 | 23:54 |
| mriedem | because MediumText is a mysql variant as defined in the migration | 23:54 |
| mtreinish | mriedem: that's easy to workaround in the test (assuming that's behavior you actually want) | 23:55 |
| mtreinish | which I'm not sure why it would be | 23:56 |
| openstackgerrit | Tracy Jones proposed openstack/nova: VMware: Refactor the image transfer https://review.openstack.org/281134 | 23:56 |
| mriedem | yeah we can make the check conditional on the db engine | 23:56 |
| mriedem | but i'm not sure if that's right | 23:56 |
| *** thorst has quit IRC | 23:57 | |
| mriedem | idk, that seems to be what we did for the other migrations that use mediumtext like this https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/migrate_repo/versions/216_havana.py#L32 | 23:58 |
| mriedem | there are no backend specific checks for those | 23:58 |
| mriedem | i don't know what the difference is for that between mysql and pg | 23:58 |
| mriedem | http://docs.sqlalchemy.org/en/latest/core/custom_types.html?highlight=with_variant#sqlalchemy.types.TypeDecorator.with_variant | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!