| *** f10d4 has quit IRC | 00:22 | |
| *** mriedem has joined #openstack-nova | 00:26 | |
| *** mriedem has quit IRC | 00:27 | |
| *** felix23ma has quit IRC | 00:29 | |
| *** vishwanathj has joined #openstack-nova | 00:35 | |
| *** felix23ma has joined #openstack-nova | 00:40 | |
| *** thorst has joined #openstack-nova | 00:40 | |
| *** hoangcx has joined #openstack-nova | 00:45 | |
| *** thorst has quit IRC | 00:47 | |
| *** thorst has joined #openstack-nova | 00:47 | |
| *** thorst has quit IRC | 00:47 | |
| *** ssurana has joined #openstack-nova | 00:48 | |
| *** kaisers1 has joined #openstack-nova | 00:51 | |
| *** kaisers has quit IRC | 00:51 | |
| *** kaisers has joined #openstack-nova | 00:52 | |
| *** kaisers2 has quit IRC | 00:52 | |
| *** markvoelker has joined #openstack-nova | 00:53 | |
| *** thorst has joined #openstack-nova | 00:54 | |
| *** lixiaoy1 has joined #openstack-nova | 01:04 | |
| *** thorst has quit IRC | 01:05 | |
| openstackgerrit | Maciej Szankin proposed openstack/nova: Deprecate all topic conf options https://review.openstack.org/351608 | 01:06 |
|---|---|---|
| *** matias has quit IRC | 01:11 | |
| *** yamahata has joined #openstack-nova | 01:21 | |
| *** READ10 has joined #openstack-nova | 01:24 | |
| *** kevinz has joined #openstack-nova | 01:28 | |
| *** hoangcx has quit IRC | 01:32 | |
| *** wangqun has joined #openstack-nova | 01:43 | |
| *** liuyulong_ is now known as liuyulong | 01:48 | |
| *** baoli has joined #openstack-nova | 01:51 | |
| *** baoli_ has joined #openstack-nova | 01:53 | |
| *** ijw has joined #openstack-nova | 01:55 | |
| *** akshai has joined #openstack-nova | 01:56 | |
| *** baoli has quit IRC | 01:56 | |
| openstackgerrit | Chen Fan proposed openstack/nova: pci: nova fail to boot when wrong setting in pci_whitelist https://review.openstack.org/357604 | 01:58 |
| *** code-R has joined #openstack-nova | 02:00 | |
| *** ijw has quit IRC | 02:01 | |
| *** EinstCrazy has joined #openstack-nova | 02:01 | |
| *** baoli_ has quit IRC | 02:03 | |
| *** code-R_ has joined #openstack-nova | 02:03 | |
| *** wangqun has quit IRC | 02:03 | |
| *** hoangcx has joined #openstack-nova | 02:03 | |
| *** wangqun_ has joined #openstack-nova | 02:03 | |
| *** ijw has joined #openstack-nova | 02:03 | |
| *** ijw has quit IRC | 02:04 | |
| *** ijw has joined #openstack-nova | 02:04 | |
| *** thorst has joined #openstack-nova | 02:05 | |
| *** phuongnh has joined #openstack-nova | 02:05 | |
| *** code-R has quit IRC | 02:06 | |
| *** ijw has quit IRC | 02:10 | |
| openstackgerrit | Nguyen Phuong An proposed openstack/nova: [api-ref]: Correcting server_groups_list parameter's type. https://review.openstack.org/365554 | 02:13 |
| *** thorst has quit IRC | 02:13 | |
| *** armax has quit IRC | 02:13 | |
| *** junjie has quit IRC | 02:14 | |
| *** code-R_ has quit IRC | 02:17 | |
| *** rfolco has quit IRC | 02:18 | |
| *** agireud has quit IRC | 02:18 | |
| *** kevinz has quit IRC | 02:18 | |
| *** kevinz has joined #openstack-nova | 02:19 | |
| *** agireud has joined #openstack-nova | 02:25 | |
| *** akshai_ has joined #openstack-nova | 02:29 | |
| *** salv-orl_ has joined #openstack-nova | 02:29 | |
| *** agireud has quit IRC | 02:30 | |
| *** akshai has quit IRC | 02:32 | |
| *** salv-orlando has quit IRC | 02:32 | |
| *** RuiChen has joined #openstack-nova | 02:33 | |
| *** annp has joined #openstack-nova | 02:34 | |
| *** ijw has joined #openstack-nova | 02:36 | |
| *** thorst has joined #openstack-nova | 02:40 | |
| *** agireud has joined #openstack-nova | 02:40 | |
| *** ijw has quit IRC | 02:40 | |
| *** vishwanathj is now known as vishwanathj_zzz | 02:47 | |
| *** amotoki has joined #openstack-nova | 02:48 | |
| *** bkopilov has quit IRC | 02:50 | |
| *** amotoki has quit IRC | 02:50 | |
| openstackgerrit | Li Yingjun proposed openstack/nova: Adds view builders for hypervisors controller https://review.openstack.org/335282 | 02:51 |
| *** akshai_ has quit IRC | 02:52 | |
| *** markvoelker has quit IRC | 02:53 | |
| *** ijw has joined #openstack-nova | 03:00 | |
| *** sudipto has joined #openstack-nova | 03:00 | |
| *** sudipto_ has joined #openstack-nova | 03:00 | |
| *** hoangcx has quit IRC | 03:01 | |
| *** houming has joined #openstack-nova | 03:04 | |
| *** ijw has quit IRC | 03:05 | |
| *** amotoki has joined #openstack-nova | 03:07 | |
| *** hoangcx has joined #openstack-nova | 03:12 | |
| *** ssurana has quit IRC | 03:13 | |
| *** amotoki has quit IRC | 03:18 | |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Stop using mox from compute/test_volumes.py https://review.openstack.org/323257 | 03:40 |
| *** READ10 has quit IRC | 03:42 | |
| *** ijw has joined #openstack-nova | 03:44 | |
| *** adisky has joined #openstack-nova | 03:47 | |
| *** ijw has quit IRC | 03:48 | |
| openstackgerrit | Jianghua Wang proposed openstack/nova: XenAPI: resolve the fetch_bandwidth failure https://review.openstack.org/365838 | 03:50 |
| *** diga has joined #openstack-nova | 03:51 | |
| *** coolsvap_ has joined #openstack-nova | 03:52 | |
| *** amotoki has joined #openstack-nova | 03:54 | |
| openstackgerrit | Eli Qiao proposed openstack/nova: Libvirt: Correct PERF_EVENTS_CPU_FLAG_MAPPING https://review.openstack.org/365842 | 03:54 |
| *** thorst has quit IRC | 03:58 | |
| *** thorst has joined #openstack-nova | 03:58 | |
| *** ijw has joined #openstack-nova | 04:00 | |
| *** thorst has quit IRC | 04:00 | |
| *** prateek has joined #openstack-nova | 04:01 | |
| *** Sukhdev has joined #openstack-nova | 04:01 | |
| *** prateek has quit IRC | 04:03 | |
| *** thorst has joined #openstack-nova | 04:03 | |
| *** sdake has joined #openstack-nova | 04:05 | |
| *** sudipto_ has quit IRC | 04:06 | |
| *** sudipto has quit IRC | 04:06 | |
| *** thorst has quit IRC | 04:08 | |
| *** ijw has quit IRC | 04:09 | |
| *** thorst has joined #openstack-nova | 04:10 | |
| *** lixiaoy1 has quit IRC | 04:12 | |
| *** thorst has quit IRC | 04:15 | |
| *** bkopilov has joined #openstack-nova | 04:24 | |
| *** sdake has quit IRC | 04:25 | |
| *** ijw has joined #openstack-nova | 04:30 | |
| murali__ | Hello | 04:31 |
| murali__ | Someone have a look https://ask.openstack.org/en/question/96444/nova-docker-issue-in-launching-the-container/ | 04:31 |
| *** moshele has joined #openstack-nova | 04:34 | |
| *** links has joined #openstack-nova | 04:39 | |
| *** ijw has quit IRC | 04:40 | |
| *** ijw has joined #openstack-nova | 04:41 | |
| *** ijw has quit IRC | 04:42 | |
| *** harlowja_at_home has joined #openstack-nova | 04:42 | |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Stop using mox in compute/test_neutron_security_groups.py https://review.openstack.org/323093 | 04:44 |
| *** harlowja_at_home has quit IRC | 04:47 | |
| *** lixiaoy1 has joined #openstack-nova | 04:47 | |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Stop using mox in unit/fake_processutils.py https://review.openstack.org/324239 | 04:48 |
| *** sdake has joined #openstack-nova | 04:50 | |
| *** ekuris has joined #openstack-nova | 04:51 | |
| *** sudipto has joined #openstack-nova | 04:53 | |
| *** sudipto_ has joined #openstack-nova | 04:53 | |
| *** dikonoor has joined #openstack-nova | 04:55 | |
| *** thorst has joined #openstack-nova | 04:55 | |
| *** su_zhang has joined #openstack-nova | 04:57 | |
| *** thorst has quit IRC | 04:59 | |
| *** sdake has quit IRC | 05:01 | |
| *** irenab has joined #openstack-nova | 05:04 | |
| *** oanson has joined #openstack-nova | 05:05 | |
| *** irenab_ has joined #openstack-nova | 05:06 | |
| *** dikonoor has quit IRC | 05:06 | |
| *** thorst has joined #openstack-nova | 05:08 | |
| *** irenab has quit IRC | 05:08 | |
| *** irenab_ is now known as irenab | 05:08 | |
| *** sdake has joined #openstack-nova | 05:10 | |
| *** ratailor has joined #openstack-nova | 05:11 | |
| *** mamandle has joined #openstack-nova | 05:11 | |
| *** wangqun_ has quit IRC | 05:11 | |
| *** wangqun_ has joined #openstack-nova | 05:11 | |
| *** sdake has quit IRC | 05:17 | |
| *** diogogmt has quit IRC | 05:19 | |
| *** su_zhang has quit IRC | 05:21 | |
| *** jaosorior has joined #openstack-nova | 05:21 | |
| *** su_zhang has joined #openstack-nova | 05:22 | |
| *** thorst has quit IRC | 05:25 | |
| *** prateek has joined #openstack-nova | 05:26 | |
| *** su_zhang has quit IRC | 05:26 | |
| *** ijw has joined #openstack-nova | 05:29 | |
| *** thorst has joined #openstack-nova | 05:29 | |
| *** zzzeek has quit IRC | 05:30 | |
| *** sridharg has joined #openstack-nova | 05:31 | |
| *** ijw_ has joined #openstack-nova | 05:31 | |
| *** nkrinner_afk is now known as nkrinner | 05:32 | |
| *** ijw has quit IRC | 05:32 | |
| *** thorst has quit IRC | 05:33 | |
| *** ijw_ has quit IRC | 05:35 | |
| *** ijw has joined #openstack-nova | 05:38 | |
| *** EinstCrazy has quit IRC | 05:40 | |
| *** ijw has quit IRC | 05:40 | |
| *** diogogmt has joined #openstack-nova | 05:41 | |
| *** liverpooler has quit IRC | 05:42 | |
| *** ijw has joined #openstack-nova | 05:43 | |
| *** lixiaoy1 has quit IRC | 05:45 | |
| *** lixiaoy11 has joined #openstack-nova | 05:45 | |
| *** ijw has quit IRC | 05:46 | |
| *** sudipto has quit IRC | 05:47 | |
| *** sudipto_ has quit IRC | 05:47 | |
| *** ijw has joined #openstack-nova | 05:47 | |
| *** hoonetorg has quit IRC | 05:48 | |
| *** mjura has joined #openstack-nova | 05:51 | |
| *** EinstCrazy has joined #openstack-nova | 05:52 | |
| *** sudipto has joined #openstack-nova | 05:53 | |
| openstackgerrit | Nguyen Phuong An proposed openstack/nova: Update info of machine-processable description https://review.openstack.org/365426 | 05:55 |
| *** unicell1 has joined #openstack-nova | 05:55 | |
| *** unicell has quit IRC | 05:56 | |
| *** ijw has quit IRC | 05:57 | |
| *** ijw has joined #openstack-nova | 05:57 | |
| murali__ | Someone have a look here : https://ask.openstack.org/en/question/96444/nova-docker-issue-in-launching-the-container/ | 05:58 |
| murali__ | I am checking on this issue now | 05:58 |
| *** mjura has quit IRC | 05:58 | |
| *** mjura has joined #openstack-nova | 05:59 | |
| openstackgerrit | huan proposed openstack/nova: xenapi: support the hotplug of a neutron port https://review.openstack.org/365505 | 06:00 |
| *** ijw has quit IRC | 06:01 | |
| *** rcarrillocruz has quit IRC | 06:02 | |
| *** ijw has joined #openstack-nova | 06:02 | |
| *** ijw has quit IRC | 06:03 | |
| openstackgerrit | Eli Qiao proposed openstack/nova: Libvirt: Correct PERF_EVENTS_CPU_FLAG_MAPPING https://review.openstack.org/365842 | 06:03 |
| *** paul-carlton has joined #openstack-nova | 06:04 | |
| *** flip214 has left #openstack-nova | 06:09 | |
| *** ijw has joined #openstack-nova | 06:10 | |
| *** hoonetorg has joined #openstack-nova | 06:12 | |
| *** raunak has joined #openstack-nova | 06:14 | |
| *** ttrumm has joined #openstack-nova | 06:21 | |
| *** EinstCrazy has quit IRC | 06:21 | |
| *** sean-k-m1 has joined #openstack-nova | 06:22 | |
| *** EinstCrazy has joined #openstack-nova | 06:23 | |
| *** sean-k-mooney has quit IRC | 06:24 | |
| slaweq_ | mdbooth: hello, I pushed yesterday clean-up patch which we talked about: https://review.openstack.org/#/c/365756/ | 06:25 |
| slaweq_ | please take a look if You will have time | 06:25 |
| *** ijw has quit IRC | 06:25 | |
| *** dimtruck is now known as zz_dimtruck | 06:27 | |
| *** ihrachys has joined #openstack-nova | 06:29 | |
| *** claudiub|2 has joined #openstack-nova | 06:29 | |
| *** claudiub has joined #openstack-nova | 06:31 | |
| *** raunak_ has joined #openstack-nova | 06:34 | |
| *** raunak has quit IRC | 06:34 | |
| *** raunak_ is now known as raunak | 06:34 | |
| *** claudiub|2 has quit IRC | 06:34 | |
| *** felix23ma has quit IRC | 06:42 | |
| *** felix23ma has joined #openstack-nova | 06:43 | |
| *** andreas_s has joined #openstack-nova | 06:45 | |
| *** zz_dimtruck is now known as dimtruck | 06:48 | |
| *** pcaruana has joined #openstack-nova | 06:48 | |
| *** felix23ma has quit IRC | 06:54 | |
| *** markvoelker has joined #openstack-nova | 06:54 | |
| *** dimtruck is now known as zz_dimtruck | 06:58 | |
| *** markvoelker has quit IRC | 06:59 | |
| *** tesseract- has joined #openstack-nova | 07:00 | |
| *** EinstCra_ has joined #openstack-nova | 07:00 | |
| *** EinstCrazy has quit IRC | 07:00 | |
| *** ijw has joined #openstack-nova | 07:00 | |
| *** zz_dimtruck is now known as dimtruck | 07:00 | |
| *** ijw has quit IRC | 07:00 | |
| *** ijw has joined #openstack-nova | 07:00 | |
| *** hoangcx has quit IRC | 07:01 | |
| *** felix23ma has joined #openstack-nova | 07:02 | |
| *** hoangcx has joined #openstack-nova | 07:04 | |
| *** sudipto has quit IRC | 07:05 | |
| *** wangqun has joined #openstack-nova | 07:05 | |
| *** wangqun_ has quit IRC | 07:06 | |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Stop using mox in api/openstack/compute/test_floating_ips.py https://review.openstack.org/322668 | 07:06 |
| *** Sukhdev has quit IRC | 07:06 | |
| *** Sukhdev has joined #openstack-nova | 07:06 | |
| *** EinstCra_ has quit IRC | 07:09 | |
| *** matrohon has joined #openstack-nova | 07:09 | |
| *** dimtruck is now known as zz_dimtruck | 07:10 | |
| *** liverpooler has joined #openstack-nova | 07:10 | |
| *** sahid has joined #openstack-nova | 07:10 | |
| *** Sukhdev has quit IRC | 07:11 | |
| *** ihrachys has quit IRC | 07:12 | |
| *** ihrachys has joined #openstack-nova | 07:12 | |
| *** slaweq__ has joined #openstack-nova | 07:14 | |
| *** ttrumm has quit IRC | 07:14 | |
| *** lixiaoy11 has quit IRC | 07:14 | |
| *** ttrumm has joined #openstack-nova | 07:15 | |
| *** lixiaoy1 has joined #openstack-nova | 07:16 | |
| *** EinstCrazy has joined #openstack-nova | 07:17 | |
| *** slaweq_ has quit IRC | 07:17 | |
| *** raunak has quit IRC | 07:18 | |
| ratailor | openstack-nova, openstack pl review https://review.openstack.org/#/c/312887/ | 07:21 |
| *** ihrachys has quit IRC | 07:23 | |
| *** ljxiash has joined #openstack-nova | 07:24 | |
| *** chrichip has quit IRC | 07:25 | |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Stop using mox in api/openstack/compute/test_remote_consoles.py https://review.openstack.org/323102 | 07:28 |
| *** felix23ma has quit IRC | 07:32 | |
| *** ijw has quit IRC | 07:35 | |
| *** gszasz has joined #openstack-nova | 07:35 | |
| *** ijw has joined #openstack-nova | 07:36 | |
| *** mamandle has quit IRC | 07:39 | |
| *** felix23ma has joined #openstack-nova | 07:41 | |
| prateek | chal | 07:43 |
| *** zenoway has joined #openstack-nova | 07:47 | |
| *** daemontool has joined #openstack-nova | 07:54 | |
| *** paul-carlton has quit IRC | 07:59 | |
| *** lpetrut has joined #openstack-nova | 08:02 | |
| *** yassine has joined #openstack-nova | 08:04 | |
| *** yassine is now known as Guest65745 | 08:04 | |
| *** Guest65745 is now known as yassou | 08:05 | |
| *** lpetrut has quit IRC | 08:07 | |
| *** jistr is now known as jistr|dentist | 08:07 | |
| *** EinstCrazy has quit IRC | 08:11 | |
| openstackgerrit | Chris Dent proposed openstack/nova: [placement] Add some tests ensuring unicode resource provider name https://review.openstack.org/365688 | 08:12 |
| *** mamandle has joined #openstack-nova | 08:14 | |
| openstackgerrit | Ha Van Tu proposed openstack/nova: Remove unused parameters https://review.openstack.org/365929 | 08:15 |
| *** derekh has joined #openstack-nova | 08:15 | |
| *** lucas-dinner is now known as lucasagomes | 08:16 | |
| *** ijw has quit IRC | 08:17 | |
| *** shangxdy has joined #openstack-nova | 08:17 | |
| *** markvoelker has joined #openstack-nova | 08:19 | |
| *** code-R has joined #openstack-nova | 08:20 | |
| *** mvk has quit IRC | 08:22 | |
| *** code-R_ has joined #openstack-nova | 08:22 | |
| *** markus_z has joined #openstack-nova | 08:22 | |
| *** paul-carlton has joined #openstack-nova | 08:22 | |
| *** ihrachys has joined #openstack-nova | 08:23 | |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Stop using mox in compute/test_security_groups.py https://review.openstack.org/323122 | 08:23 |
| *** markvoelker has quit IRC | 08:24 | |
| *** hshiina has joined #openstack-nova | 08:24 | |
| *** code-R has quit IRC | 08:25 | |
| *** gongysh has quit IRC | 08:25 | |
| *** gongysh has joined #openstack-nova | 08:26 | |
| *** EinstCrazy has joined #openstack-nova | 08:27 | |
| *** Jeffrey4l has quit IRC | 08:28 | |
| *** EinstCrazy has quit IRC | 08:28 | |
| *** EinstCrazy has joined #openstack-nova | 08:28 | |
| *** mnestratov has joined #openstack-nova | 08:29 | |
| *** salv-orlando has joined #openstack-nova | 08:29 | |
| openstackgerrit | Zhenyu Zheng proposed openstack/nova: Trival fix a typo https://review.openstack.org/365938 | 08:31 |
| *** salv-orl_ has quit IRC | 08:32 | |
| *** roeyc has joined #openstack-nova | 08:32 | |
| *** sudipto has joined #openstack-nova | 08:34 | |
| *** danpawlik1 has joined #openstack-nova | 08:34 | |
| danpawlik1 | Hello, if you have some free time pls check https://review.openstack.org/#/c/314928/ :) | 08:35 |
| openstackgerrit | Maxim Nestratov proposed openstack/nova: libvirt: add supported vif types for virtuozzo virt_type https://review.openstack.org/365943 | 08:35 |
| *** ababich has joined #openstack-nova | 08:35 | |
| *** ijw has joined #openstack-nova | 08:39 | |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Stop using mox in virt/libvirt/test_imagecache.py https://review.openstack.org/323717 | 08:44 |
| *** ecelik has quit IRC | 08:47 | |
| *** ecelik has joined #openstack-nova | 08:48 | |
| *** hshiina has quit IRC | 08:50 | |
| *** mvk has joined #openstack-nova | 08:51 | |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Stop using mox in compute/test_server_actions.py https://review.openstack.org/323186 | 08:52 |
| *** lixiaoy1 has quit IRC | 08:57 | |
| *** EinstCrazy has quit IRC | 08:59 | |
| *** hoangcx has quit IRC | 08:59 | |
| *** ecelik has quit IRC | 09:00 | |
| *** ecelik has joined #openstack-nova | 09:00 | |
| *** zz_dimtruck is now known as dimtruck | 09:01 | |
| *** EinstCrazy has joined #openstack-nova | 09:01 | |
| *** ecelik has quit IRC | 09:04 | |
| *** ecelik has joined #openstack-nova | 09:04 | |
| *** ijw has quit IRC | 09:07 | |
| *** rwmjones has quit IRC | 09:08 | |
| *** dimtruck is now known as zz_dimtruck | 09:10 | |
| *** rwmjones has joined #openstack-nova | 09:12 | |
| *** danpb has joined #openstack-nova | 09:16 | |
| *** ijw has joined #openstack-nova | 09:18 | |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Stop using mox int unit/virt/xenapi/image/test_utils.py https://review.openstack.org/342626 | 09:18 |
| *** gongysh has quit IRC | 09:19 | |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Stop using mox in virt/xenapi/image/test_vdi_through_dev.py https://review.openstack.org/342682 | 09:21 |
| *** gongysh has joined #openstack-nova | 09:21 | |
| *** jistr|dentist is now known as jistr | 09:23 | |
| *** mfedosin has joined #openstack-nova | 09:25 | |
| *** gongysh has quit IRC | 09:26 | |
| *** daemontool has quit IRC | 09:29 | |
| *** lpetrut has joined #openstack-nova | 09:30 | |
| *** lpetrut has quit IRC | 09:30 | |
| *** lpetrut has joined #openstack-nova | 09:31 | |
| *** shangxdy has quit IRC | 09:31 | |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Stop using mox in unit/virt/xenapi/test_vm_utils.py (1) https://review.openstack.org/344054 | 09:35 |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Stop using mox in unit/virt/test_block_device.py https://review.openstack.org/342595 | 09:37 |
| *** roeyc has quit IRC | 09:38 | |
| *** mvk has quit IRC | 09:38 | |
| *** cdent has joined #openstack-nova | 09:39 | |
| *** PaulMurray has joined #openstack-nova | 09:39 | |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Stop using mox in unit/cells/test_cells_rpc_driver.py https://review.openstack.org/333101 | 09:39 |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Stop using mox int unit/virt/xenapi/stubs.py https://review.openstack.org/334848 | 09:43 |
| *** sdake has joined #openstack-nova | 09:45 | |
| openstackgerrit | Dao Cong Tien proposed openstack/nova: Stop using mox in unit/virt/xenapi/test_vmops.py (2/2) https://review.openstack.org/344010 | 09:46 |
| *** zenoway has quit IRC | 09:46 | |
| *** zenoway has joined #openstack-nova | 09:47 | |
| *** zenoway has quit IRC | 09:51 | |
| *** sean-k-m1 is now known as sean-k-mooney | 09:53 | |
| *** annp has quit IRC | 09:57 | |
| *** mvk has joined #openstack-nova | 09:58 | |
| *** mnestratov has quit IRC | 10:00 | |
| *** zenoway has joined #openstack-nova | 10:00 | |
| *** zenoway has quit IRC | 10:01 | |
| *** zenoway has joined #openstack-nova | 10:01 | |
| openstackgerrit | Ghanshyam Mann proposed openstack/nova: Body Verification of os-aggregates.inc https://review.openstack.org/366023 | 10:03 |
| *** zz_dimtruck is now known as dimtruck | 10:06 | |
| *** Sukhdev has joined #openstack-nova | 10:07 | |
| mdbooth | ftersin: Sorry, you got a load of my frustration this morning :) Thanks for all the reviews. | 10:11 |
| * mdbooth tries to keep it bottled, but doesn't always quite succeed. | 10:12 | |
| mdbooth | That's healthy, right? | 10:12 |
| *** Sukhdev has quit IRC | 10:13 | |
| *** phuongnh has quit IRC | 10:14 | |
| *** liuyulong is now known as liuyulong|away | 10:15 | |
| *** riccardo has joined #openstack-nova | 10:16 | |
| ftersin | bdbooth: just found a bug of ephemeral resizing, went to find a solution, remembered that you discussed it here, and start to investigate the current state of your work | 10:16 |
| *** roeyc has joined #openstack-nova | 10:18 | |
| mdbooth | ftersin: Yeah, hasn't worked ever, I think | 10:18 |
| mdbooth | That patch half fixes it. I think I note that it's not fixed in one of the unit tests, don't I? | 10:19 |
| *** ijw has quit IRC | 10:20 | |
| mdbooth | Incidentally, remember that 'disk.local' is never created for new instances, and hasn't been for releases covering several years at least. | 10:20 |
| riccardo | hi guys, when I google for something like, how to connect to DB, it always displays results for openstack administrators. How can I search for contributer/programmer stuff? Any idea? | 10:20 |
| mdbooth | Which basically means that 'disk.local' is rare in the wild, and quite possibly extinct | 10:20 |
| mdbooth | A single ephemeral disks created today is 'disk.eph0' | 10:21 |
| ftersin | 'disk.local' is a clear example of the unit tests quality :/ | 10:22 |
| mdbooth | Yup | 10:22 |
| ftersin | ephN is used since bdm v2 | 10:22 |
| mdbooth | Indeed | 10:22 |
| openstackgerrit | Rajesh Tailor proposed openstack/nova: Fix instance delete stuck in deleting task_state https://review.openstack.org/294491 | 10:24 |
| *** sahid has quit IRC | 10:28 | |
| ftersin | mdbooth: "That patch half fixes it" - which patch do you mean? | 10:28 |
| mdbooth | You commented on it in a review one of my patches | 10:28 |
| mdbooth | It's one of the _create_image patches | 10:29 |
| *** kaisers2 has joined #openstack-nova | 10:29 | |
| *** sahid has joined #openstack-nova | 10:29 | |
| *** kaisers1 has quit IRC | 10:29 | |
| *** kaisers1 has joined #openstack-nova | 10:30 | |
| mdbooth | ftersin: This one: https://review.openstack.org/#/c/331118/15/nova/virt/libvirt/driver.py | 10:30 |
| *** Oku_OS is now known as Oku_OS-away | 10:30 | |
| *** kaisers has quit IRC | 10:30 | |
| *** Oku_OS-away is now known as Oku_OS | 10:31 | |
| mdbooth | Half fixed == not fixed, of course :) | 10:32 |
| * mdbooth is just trying to find the code which mangles bdms in that call path. I've found it before, but it's currently eluding me. | 10:32 | |
| *** ecelik has quit IRC | 10:32 | |
| mdbooth | It's either in compute api, conductor, or compute manager. | 10:32 |
| *** PaulMurray has quit IRC | 10:33 | |
| ftersin | mdboot: well, i think that if nova with this fix can resize or add a single ephemeral, we may amend the case with numerous ephemerals | 10:33 |
| mdbooth | Anyway, the instance we're passed has had its metadata resized, but its bdms have not been resized. | 10:33 |
| *** sdake has quit IRC | 10:33 | |
| mdbooth | Because the resize code uses bdms, it means we resize, but to the same size as before | 10:33 |
| mdbooth | Anyway, with that change it will at least iterate over ephemerals, but it'll uselessly resize them to their existing size. | 10:34 |
| mdbooth | We need to also fix the caller. | 10:34 |
| mdbooth | I'm determined to find that code again :) | 10:34 |
| ftersin | if your series is blocked by newton code freeze, does it make sence to push https://review.openstack.org/#/c/346263 at the moment? | 10:35 |
| *** ttrumm_ has joined #openstack-nova | 10:35 | |
| *** zenoway has quit IRC | 10:36 | |
| mdbooth | ftersin: I have been +1 on several patches already which will conflict with my series for exactly that reason. | 10:36 |
| mdbooth | Yes, expedient bug fixes are always good. | 10:36 |
| *** zenoway has joined #openstack-nova | 10:36 | |
| *** ecelik has joined #openstack-nova | 10:37 | |
| mdbooth | They're also easier to backport. | 10:37 |
| mdbooth | ftersin: I've already reviewed that patch and was in favour of it, but it needs a test. | 10:38 |
| *** roeyc has quit IRC | 10:38 | |
| slaweq__ | mdbooth: hello | 10:38 |
| *** wangqun has quit IRC | 10:38 | |
| slaweq__ | mdbooth: do You know who from core team I could ask to check https://review.openstack.org/#/c/363926/ ? | 10:38 |
| *** tbachman has quit IRC | 10:39 | |
| *** ttrumm has quit IRC | 10:39 | |
| ftersin | mdboot: but you just sayed that it requires bdm update as well, right? | 10:39 |
| mdbooth | Anybody you can get :) Bug fixes are the priority right now, so hopefully you'll get some attention. | 10:39 |
| *** code-R_ has quit IRC | 10:39 | |
| ftersin | so that not test only, but some other updates... | 10:40 |
| mdbooth | ftersin: Ah, yes, I didn't read the whole context. Yes, I don't think that will adress the whole problem. | 10:40 |
| *** zenoway has quit IRC | 10:40 | |
| *** ecelik has quit IRC | 10:41 | |
| ftersin | i've asked cfriesen, if he wants to push it to newton and legacy releases. waiting for an answer | 10:41 |
| ftersin | ok, thanks | 10:41 |
| *** cdent has quit IRC | 10:42 | |
| openstackgerrit | Alberto Planas proposed openstack/nova: Add migrate_uri for invoking the migration https://review.openstack.org/356558 | 10:43 |
| *** zenoway has joined #openstack-nova | 10:44 | |
| *** PaulMurray has joined #openstack-nova | 10:44 | |
| *** roeyc has joined #openstack-nova | 10:45 | |
| *** HenryG_ is now known as HenryG | 10:46 | |
| *** lyarwood is now known as lyarwood_ | 10:46 | |
| *** baoli has joined #openstack-nova | 10:46 | |
| *** lyarwood_ is now known as lyarwood | 10:46 | |
| *** esracelik has joined #openstack-nova | 10:47 | |
| *** zengine has quit IRC | 10:48 | |
| mdbooth | ftersin: It's in ComputeManager._finish_resize | 10:49 |
| mdbooth | This is where compute manager resizes the instance's metadata | 10:49 |
| mdbooth | However, it doesn't update bdms, which are the only thing we can reasonably look at for ephemerals | 10:50 |
| mdbooth | We need to also poke bdms in there before calling driver.finish_migration() | 10:50 |
| *** _gryf has quit IRC | 10:50 | |
| *** zzzeek has joined #openstack-nova | 10:51 | |
| mdbooth | With that, and the fix the both cfriesen and I have done independently, resize of ephemerals should work. | 10:51 |
| *** _gryf has joined #openstack-nova | 10:51 | |
| ftersin | but whether we must distinguish between bdms added implicitly by flavor and explicitly by user/snapshot? | 10:52 |
| sfinucan | mriedem: When you're online, could I get the -2 removed from this centralize-config-options patch? https://review.openstack.org/#/c/357061/ | 10:53 |
| ftersin | i.e. if a user run instance on a flavor with 20 GB ephemeral, specified 10 GB only in bdm, and then resize the instance to 40 GB ephemeral flavor, what to do with the ephemeral? | 10:54 |
| *** EinstCrazy has quit IRC | 10:54 | |
| mdbooth | danpawlik1: I'd recommend against block adding core reviewers, btw. You risk annoying them :/ | 10:55 |
| danpb | indeed, reviews with people block-added go to the bottom of my priority list :-) | 10:55 |
| mdbooth | ftersin: Whatever we do, that's nothing to do with the driver. | 10:56 |
| *** zzzeek has quit IRC | 10:56 | |
| mdbooth | But yes, that's a good question. There's also: | 10:56 |
| danpawlik1 | mdbooth: sorry for that | 10:56 |
| mdbooth | Flavor specifies 20GB ephemeral, user adds 2 ephemerals 10GB each. | 10:56 |
| mdbooth | User resizes to flavor with 40GB ephemeral. What does the resize do? | 10:57 |
| ftersin | mdbooth: this is another case - how to split inreased space onto exising ephemerals | 10:57 |
| *** zzzeek has joined #openstack-nova | 10:57 | |
| danpb | mdbooth: i'd just split it proportionally across the disks | 10:57 |
| *** baoli has quit IRC | 10:57 | |
| danpb | there's nothing much else you can do that's sane, given the API limitations | 10:57 |
| riccardo | hi guys, when I google for something like, "how to connect to DB", it always displays results for openstack administrators. How can I search for contributer/programmer stuff? Any idea? | 10:57 |
| mdbooth | ftersin: We discussed this incredibly briefly ages ago, and the answer at the time was: it's always been broken anyway, so don't sweat it. Just fix the case where there's a single, full size ephemeral disk and leave anything else for a future design change. | 10:58 |
| *** kaisers2 has quit IRC | 10:58 | |
| mdbooth | However, ephemerals, especially multiple ephemerals, have been broken for so long in so many ways, I'm not personally convinced anybody's using them. | 10:58 |
| *** kaisers1 has quit IRC | 10:59 | |
| mdbooth | danpb: I think it's safer to leave it alone in that case, tbh. | 10:59 |
| *** kaisers has joined #openstack-nova | 10:59 | |
| *** kaisers1 has joined #openstack-nova | 10:59 | |
| *** baoli has joined #openstack-nova | 10:59 | |
| mdbooth | Seems like a lot of hassle for an unused feature. | 10:59 |
| *** baoli has quit IRC | 11:01 | |
| openstackgerrit | Claudiu Belu proposed openstack/nova-specs: Adds spec for instance live resize https://review.openstack.org/141219 | 11:02 |
| ftersin | mdbooth: but if a feature is broken, who uses it? how customers do live now with no ephemeral resize? do workaround via snapshots? | 11:03 |
| mdbooth | That's the thing, maybe they don't use it. What's the use case for multiple ephemerals anyway? | 11:04 |
| mdbooth | Why would a user ever request an ephemeral smaller than what they've paid for? | 11:04 |
| *** Jeffrey4l has joined #openstack-nova | 11:05 | |
| mdbooth | I wonder if we've designed features which users don't need or use. | 11:05 |
| danpb | mdbooth: i can see people wanting separate disks for separate usage scenarios | 11:05 |
| *** cdent has joined #openstack-nova | 11:05 | |
| ftersin | i would want to split if i wanted to separate fast space between different services or data purposes | 11:06 |
| danpb | mdbooth: eg, one might contain /var/www/ while the other contained /home | 11:06 |
| mdbooth | danpb: Maybe if their workload uses raw disk? | 11:06 |
| danpb | yes you could use partitions, but using separate disks means you avoid contention in the i.o stack so theoretically could get better performance | 11:06 |
| danpb | i agree though that use separate disks is a niche use case | 11:06 |
| mdbooth | It's still the same disk on the backend :) | 11:07 |
| danpb | mdbooth: that's a mere impl detail | 11:07 |
| mdbooth | So qemu might use a slightly different path, but the spinning rust/magical cloud bits are the same. | 11:07 |
| danpb | from an API pov it makes sense | 11:07 |
| ftersin | i would want to limit my ephemeral, if i did not need it so much, but needed vcpus for example, and given flavor set did not give me a big choice, but i needed to honor disk quota | 11:07 |
| danpb | its just that our impl isn't taking advantage of it | 11:07 |
| danpb | mdbooth: you will at least have separate i/o queues in the guest os side which can improve performance in a multi vcpu guest | 11:08 |
| mdbooth | danpb: That's somewhere I wanted to go after persistent storage metadata, btw. That will allow us to specify different backends for different disks. | 11:08 |
| mdbooth | Right now, though, there's nothing. | 11:08 |
| danpb | yeah, people have requested separate backends for local disks before | 11:08 |
| mdbooth | ftersin: Anyway, I'm not disagreeing with you, just expounding the virtues of expediently fixing the thing with an obvious fix, and leaving the rest for another time. | 11:10 |
| mdbooth | ftersin: How important is ephemeral resize to you, btw? | 11:10 |
| mdbooth | Have you got users complaining about this? | 11:10 |
| ftersin | that's thing i agree too | 11:11 |
| *** PaulMurray has quit IRC | 11:11 | |
| ftersin | i'm blocked with this on scalio driver | 11:11 |
| mdbooth | Ephemeral resize? | 11:11 |
| ftersin | not directly, but yes | 11:12 |
| mdbooth | Well, it doesn't work with any other driver, so why should it block you? | 11:12 |
| ftersin | scaleio volumes need to be mapped to a compute host | 11:12 |
| danpb | yeah, i'm not seeing why instance resize is a blocker in general | 11:12 |
| *** ljxiash has quit IRC | 11:13 | |
| ftersin | when the resizing moves the instance to another host, volumes must be remapped to that host | 11:13 |
| *** PaulMurray has joined #openstack-nova | 11:13 | |
| *** roeyc has quit IRC | 11:14 | |
| ftersin | imagebackend's create_image is the only place where i can do it | 11:14 |
| ftersin | so i need create_image to be called inside finish_migration | 11:15 |
| ftersin | for each disk, including ephemerals | 11:15 |
| mdbooth | I see no problem with that. | 11:15 |
| ftersin | even if size of ephemeral is not changed | 11:15 |
| mdbooth | Does cfriesen's patch fix it for you? | 11:15 |
| ftersin | hope for that, but have not checked yet | 11:16 |
| mdbooth | However, I think that's the wrong place to do that. | 11:16 |
| mdbooth | I'm going to open a can of worms here, stay with me, but this is also broken for Rbd. | 11:17 |
| mdbooth | Which, iiuc, is the same model. | 11:17 |
| mdbooth | i.e. remote storage, which we're going to connect to from the new host and not actually copy. Right? | 11:17 |
| ftersin | rbd does not require mapping operation, though | 11:17 |
| mdbooth | Right, but it does exclude it from the copy. | 11:17 |
| mdbooth | Which I'm assuming you also have to be careful to do. | 11:18 |
| ftersin | which copy operation you mean? | 11:18 |
| mdbooth | migrate_disk_and_power_off | 11:18 |
| mdbooth | You'll presumably need to ensure that your scaleio disks aren't copied there if they're just going to be moved | 11:18 |
| mdbooth | Involving some kind of shared storage negotiation | 11:18 |
| *** cdelatte has joined #openstack-nova | 11:19 | |
| mdbooth | So, for scaleio do you need to attach the disks to the *host*? | 11:19 |
| ftersin | rbd and scaleio are shared block storage, no one need to copy data of them | 11:19 |
| mdbooth | So, for Rbd there's no host operation required because qemu talks rbd directly | 11:20 |
| mdbooth | But iiuc, the scaleio thing is more akin to attaching a volume | 11:20 |
| mdbooth | Is it persistent across host reboots? | 11:20 |
| ftersin | moreover, a lot of driver code does copy and other things with files, which comes from libvirt domain via get_instance_disk_info | 11:20 |
| ftersin | but rbd devices are not returned from there | 11:21 |
| mdbooth | Yeah, that's gross. | 11:21 |
| ftersin | i did the same for scaleio ones | 11:21 |
| mdbooth | That's how they're exluded from the copy. | 11:21 |
| mdbooth | I'm just specifically thinking about the attach thing. | 11:21 |
| mdbooth | If it's not persistent across host reboots, then you can't do it in create_image anyway. | 11:22 |
| *** ljxiash has joined #openstack-nova | 11:22 | |
| ftersin | yes. otherwise it do not work at all | 11:22 |
| mdbooth | You need another hook. | 11:22 |
| *** lucasagomes is now known as lucas-hungry | 11:22 | |
| ftersin | scaleio mapping is persistent | 11:22 |
| *** rtheis has joined #openstack-nova | 11:22 | |
| mdbooth | Ok, cool. | 11:23 |
| mdbooth | So, now I think I understand your comment about ensure_backing. | 11:23 |
| *** roeyc has joined #openstack-nova | 11:23 | |
| mdbooth | Because that's basically exactly the same hook you want for scalio. | 11:23 |
| mdbooth | Namely, don't copy anything, just ensure the disk exists at the remote end. | 11:24 |
| *** bkopilov has quit IRC | 11:27 | |
| *** ijw has joined #openstack-nova | 11:27 | |
| riccardo | how can i read and write to MariaDB? Is there some code in oslo or do I need to write everything from the scratch? | 11:29 |
| ftersin | but.. since i already added hooks to map and unmap all volumes, probably it is better just to add another hook to map all volumes in finish_migration | 11:29 |
| *** amotoki_ has joined #openstack-nova | 11:29 | |
| *** ijw has quit IRC | 11:29 | |
| *** ijw has joined #openstack-nova | 11:30 | |
| *** amotoki has quit IRC | 11:31 | |
| ftersin | mdbooth: exactly. i added a hook to map all instance disks to pre_live_migration, so i just add the same to finish_migration, and do not worry about resizing | 11:31 |
| ftersin | excellent solution. thanks for the useful conversation | 11:32 |
| *** ljxiash has quit IRC | 11:33 | |
| ftersin | danpb: about people question. did anyone ask to have cinder as the ephemeral backend? | 11:36 |
| *** mfedosin has quit IRC | 11:37 | |
| danpb | i don't know | 11:37 |
| *** sdague has joined #openstack-nova | 11:39 | |
| riccardo | how can I read in the database how many hosts are online? | 11:39 |
| *** PaulMurray has quit IRC | 11:41 | |
| *** kaisers has quit IRC | 11:41 | |
| *** fragatina has joined #openstack-nova | 11:42 | |
| *** kaisers has joined #openstack-nova | 11:42 | |
| *** kaisers1 has quit IRC | 11:43 | |
| *** kaisers1 has joined #openstack-nova | 11:43 | |
| *** fragatina has quit IRC | 11:43 | |
| *** mamandle has quit IRC | 11:45 | |
| *** kaisers2 has joined #openstack-nova | 11:46 | |
| *** kaisers3 has joined #openstack-nova | 11:47 | |
| *** kaisers1 has quit IRC | 11:48 | |
| *** kaisers has quit IRC | 11:49 | |
| *** mfedosin has joined #openstack-nova | 11:51 | |
| *** salv-orl_ has joined #openstack-nova | 11:52 | |
| *** salv-orlando has quit IRC | 11:52 | |
| *** code-R has joined #openstack-nova | 11:54 | |
| *** sudipto has quit IRC | 11:55 | |
| *** kaisers3 has quit IRC | 11:55 | |
| *** kaisers has joined #openstack-nova | 11:55 | |
| *** code-R_ has joined #openstack-nova | 11:58 | |
| *** code-R has quit IRC | 12:01 | |
| *** daemontool has joined #openstack-nova | 12:04 | |
| *** gcb has quit IRC | 12:04 | |
| sdague | cdent: are you planning on cleaning up the log string nits in - https://review.openstack.org/#/c/365015/ ? | 12:04 |
| *** ijw has quit IRC | 12:05 | |
| cdent | sdague: yes, but I can do that later if you think we should get that in sooner than later | 12:06 |
| cdent | the number of 409s that are happening should be improved by some of the stuff above that patch | 12:06 |
| cdent | the client side doesn't have any magical generation counting so it sometimes has to 409 to know that it needs to get a new generation to rectify itself | 12:07 |
| cdent | sdague: I was waiting for the US to wake up before making any more adjustments on that stack so that they can happen in a batch. | 12:08 |
| *** tbachman has joined #openstack-nova | 12:08 | |
| sdague | cdent: ok, fair. | 12:09 |
| *** rfolco has joined #openstack-nova | 12:09 | |
| openstackgerrit | Kashyap Chamarthy proposed openstack/nova: libvirt: fix detach volume race condition https://review.openstack.org/290510 | 12:09 |
| *** daemontool_ has joined #openstack-nova | 12:10 | |
| kashyap | mdbooth: The above is just a rebase to Git first. Now reworking the feedback from reviewers | 12:10 |
| kashyap | s/Git/Git master/ | 12:10 |
| *** daemontool has quit IRC | 12:12 | |
| *** fragatina has joined #openstack-nova | 12:13 | |
| *** tbachman_ has joined #openstack-nova | 12:13 | |
| *** edmondsw has joined #openstack-nova | 12:13 | |
| sdague | cdent: even at top of stack the 409ing seems pretty high on allocations - http://logs.openstack.org/19/365719/1/check/gate-tempest-dsvm-neutron-placement-full-ubuntu-xenial-nv/a1c6cc4/logs/apache/placement-api.txt.gz | 12:14 |
| sdague | which is definitely leading to a lot of 404s on deletes, so explains why the allocations might not line up with reality | 12:14 |
| *** tbachman has quit IRC | 12:15 | |
| *** tbachman_ is now known as tbachman | 12:15 | |
| cdent | sdague: I don't think the allocations handling on the client side is very smart yet, so we'll need to do some analysis across the placement log and the n-cpu log to find out what's going on with a bit more detail. I'll suck down the n-cpu log and see if any patterns leap out, but we'll probably want Dan too. | 12:19 |
| *** rlrossit has joined #openstack-nova | 12:19 | |
| *** ttrumm_ has quit IRC | 12:19 | |
| cdent | (I've got a meeting in a few minutes that's going to delay any progress for me) | 12:20 |
| *** liverpooler has quit IRC | 12:21 | |
| sdague | yep, no prob | 12:21 |
| cdent | "Unable to allocate inventory: Unable to create allocation for 'VCPU' on resource provider '4a25cc16-0b8a-4759-a29f-c17c6650dd39'. The requested amount would exceed the capacity" makes no sense so should be a clue | 12:21 |
| *** oanson has quit IRC | 12:22 | |
| *** fragatina has quit IRC | 12:22 | |
| *** ttrumm has joined #openstack-nova | 12:24 | |
| cdent | one thing sdague looks like we still need to change things so that placement api starts before n-cpu. I think we talked about that last week, but I forgot about it. | 12:24 |
| sdague | ok, though shouldn't we be handling that in code? | 12:25 |
| sdague | I thought we were handling the connect issue and it would just mean a missed request | 12:25 |
| *** woodster_ has joined #openstack-nova | 12:26 | |
| cdent | it would, and it's fine. I don't think it is causing problems, it's just a "wasted" error. | 12:26 |
| *** PaulMurray has joined #openstack-nova | 12:26 | |
| *** clenimar has quit IRC | 12:26 | |
| *** lucas-hungry is now known as lucasagomes | 12:27 | |
| cdent | sdague: there's a 147 difference instances being allocated. is that around what we expect? do we even know? | 12:30 |
| *** thorst has joined #openstack-nova | 12:30 | |
| *** jaypipes has joined #openstack-nova | 12:30 | |
| *** clenimar has joined #openstack-nova | 12:31 | |
| *** xinliang has quit IRC | 12:32 | |
| sdague | that sounds ball park correct | 12:32 |
| sdague | I was pretty sure it was > 100 | 12:32 |
| *** mamandle has joined #openstack-nova | 12:34 | |
| sdague | cdent: which run are you looking at? | 12:35 |
| cdent | the one you pasted above | 12:35 |
| *** mdrabe has joined #openstack-nova | 12:37 | |
| cdent | that's using awks, greps, sorts uniq on the n-cpu log | 12:37 |
| *** tongli has joined #openstack-nova | 12:40 | |
| *** EinstCrazy has joined #openstack-nova | 12:43 | |
| *** xinliang has joined #openstack-nova | 12:44 | |
| *** baoli has joined #openstack-nova | 12:44 | |
| *** ttrumm has quit IRC | 12:44 | |
| *** baoli_ has joined #openstack-nova | 12:45 | |
| *** figleaf is now known as edleafe | 12:45 | |
| *** ttrumm has joined #openstack-nova | 12:46 | |
| *** baoli has quit IRC | 12:48 | |
| *** jaosorior has quit IRC | 12:48 | |
| *** jaosorior has joined #openstack-nova | 12:49 | |
| *** mmedvede has quit IRC | 12:49 | |
| *** jaosorior has quit IRC | 12:49 | |
| *** pcaruana has quit IRC | 12:50 | |
| *** tbachman_ has joined #openstack-nova | 12:51 | |
| *** ratailor has quit IRC | 12:51 | |
| *** jaosorior has joined #openstack-nova | 12:51 | |
| *** tbachman has quit IRC | 12:51 | |
| *** tbachman_ is now known as tbachman | 12:51 | |
| *** sudipto has joined #openstack-nova | 12:51 | |
| *** sridharg has quit IRC | 12:56 | |
| *** vishwanathj_zzz is now known as vishwanathj | 12:57 | |
| cdent | sdague: when I get off this call I'm going to make sure that the capacity violation tests on the api side are sufficient so we can be sure to bisect the problem properly | 12:58 |
| *** diga has quit IRC | 12:59 | |
| *** edand has joined #openstack-nova | 13:00 | |
| *** tbachman has quit IRC | 13:00 | |
| *** zengine has joined #openstack-nova | 13:00 | |
| *** vladikr has joined #openstack-nova | 13:00 | |
| *** houming has quit IRC | 13:01 | |
| *** bkopilov has joined #openstack-nova | 13:03 | |
| *** mingyu has joined #openstack-nova | 13:06 | |
| sdague | cdent: are we handling cpu overcommit fine in the placement API? | 13:07 |
| *** tbachman has joined #openstack-nova | 13:08 | |
| cdent | sdague: that's what allocation ratio means in the inventory, and what I'm about to check is that the tests for that are sufficient to be sure it is correct | 13:08 |
| cdent | then we can work back from there to where the two concepts comingle | 13:08 |
| *** sridharg has joined #openstack-nova | 13:09 | |
| *** mingyu_ has joined #openstack-nova | 13:09 | |
| openstackgerrit | Jianghua Wang proposed openstack/nova: XenAPI: resolve the fetch_bandwidth failure https://review.openstack.org/365838 | 13:10 |
| *** mmedvede has joined #openstack-nova | 13:11 | |
| *** mingyu has quit IRC | 13:12 | |
| *** mnestratov has joined #openstack-nova | 13:15 | |
| *** Guest36215 has quit IRC | 13:15 | |
| openstackgerrit | Jianghua Wang proposed openstack/nova: XenAPI: resolve the fetch_bandwidth failure https://review.openstack.org/365838 | 13:15 |
| *** akshai has joined #openstack-nova | 13:18 | |
| openstackgerrit | Alex Szarka proposed openstack/nova-specs: Typo: very instead of vary https://review.openstack.org/366125 | 13:18 |
| *** ttrumm_ has joined #openstack-nova | 13:22 | |
| *** ljxiash has joined #openstack-nova | 13:23 | |
| *** tblakes has joined #openstack-nova | 13:24 | |
| *** eharney has joined #openstack-nova | 13:25 | |
| *** ttrumm has quit IRC | 13:25 | |
| *** tblakes_ has joined #openstack-nova | 13:25 | |
| *** tbachman has quit IRC | 13:25 | |
| *** hemanthm|away is now known as hemanthm | 13:27 | |
| markus_z | bauzas: Can I interest you in a scheduler bug fix? https://review.openstack.org/#/c/352344/ | 13:29 |
| *** tblakes has quit IRC | 13:29 | |
| *** tblakes_ is now known as tblakes | 13:30 | |
| *** irenab has quit IRC | 13:31 | |
| *** pwnall1 has joined #openstack-nova | 13:32 | |
| *** tbachman has joined #openstack-nova | 13:32 | |
| *** irenab has joined #openstack-nova | 13:32 | |
| bauzas | markus_z: cool :) | 13:33 |
| *** mriedem has joined #openstack-nova | 13:34 | |
| *** mingyu_ has quit IRC | 13:34 | |
| *** zengine has quit IRC | 13:35 | |
| *** markvoelker has joined #openstack-nova | 13:36 | |
| openstackgerrit | Slawek Kaplonski proposed openstack/nova: libvirt: Clean up block device job status reporting code https://review.openstack.org/365756 | 13:36 |
| *** andymaier has joined #openstack-nova | 13:37 | |
| *** mingyu has joined #openstack-nova | 13:37 | |
| *** pwnall1 has quit IRC | 13:38 | |
| *** pwnall1 has joined #openstack-nova | 13:38 | |
| *** sridharg has quit IRC | 13:39 | |
| riccardo | hi guys, can anyone help? how can i get data from MariaDB? | 13:39 |
| *** shangxdy has joined #openstack-nova | 13:40 | |
| riccardo | is there some python example code? | 13:40 |
| openstackgerrit | MJWurtz proposed openstack/nova: Refresh info_cache after deleting floating IP https://review.openstack.org/357494 | 13:42 |
| *** ttrumm_ has quit IRC | 13:43 | |
| *** yamahata has quit IRC | 13:44 | |
| *** ttrumm has joined #openstack-nova | 13:44 | |
| openstackgerrit | Prateek Arora proposed openstack/nova: Handle ConnectFailure from keystone when neutron is down https://review.openstack.org/327706 | 13:45 |
| *** cleong has joined #openstack-nova | 13:45 | |
| *** TravT has joined #openstack-nova | 13:45 | |
| *** roeyc has quit IRC | 13:46 | |
| *** dimtruck is now known as zz_dimtruck | 13:47 | |
| *** roeyc has joined #openstack-nova | 13:48 | |
| *** ttrumm has quit IRC | 13:49 | |
| openstackgerrit | Prateek Arora proposed openstack/nova: Handle ConnectFailure from keystone when neutron is down https://review.openstack.org/327706 | 13:49 |
| prateek | mriedem, can you please have a look at https://review.openstack.org/#/c/294513/ whenever you have some time ? | 13:50 |
| openstackgerrit | Chris Dent proposed openstack/nova: [placement] Add additional allocation tests https://review.openstack.org/366144 | 13:50 |
| cdent | sdague: that ^ is more tests, but they didn't reveal anything on the server side | 13:51 |
| *** mlavalle has joined #openstack-nova | 13:51 | |
| cdent | sdague I'm going to add some more logging to the resource tracker so we can see what it is writing more clearly. Or I can put it on the server side if that seems better? | 13:51 |
| *** gongysh has joined #openstack-nova | 13:52 | |
| sdague | if it's on the server side we could also add a debug log which dumps the current allocation state of the world on every request, to see what the system thinks is there | 13:52 |
| mdbooth | ftersin: Hey, if I asked you to finish the cleanup in https://review.openstack.org/#/c/334609/, would it get done? | 13:53 |
| mdbooth | ftersin: There are 2 identical misuses of that function in driver, but you only clean up one. | 13:53 |
| mdbooth | Fixing 1 is obviously better than fixing none :) | 13:53 |
| mdbooth | But fixing both is best. | 13:54 |
| * cdent spins up devstack | 13:54 | |
| ftersin | i remove the second in the next patch | 13:54 |
| mdbooth | ftersin: Ah, ok. | 13:54 |
| mdbooth | Hadn't seen that. | 13:55 |
| *** rlrossit has quit IRC | 13:56 | |
| PaulMurray | Live migration meeting in a few minutes on #openstack-meeting-3 PaulMurray paul-carlton2 eliqiao shaohe_feng pkoniszewski jlanoux yuntongjin mdbooth kashyap kmartin claudiub tobasco tdurakov andreas_s abhishekk mrhillsman raj_singh | 13:56 |
| *** browne has joined #openstack-nova | 13:57 | |
| *** burt has joined #openstack-nova | 13:57 | |
| ftersin | mdbooth: i abandoned my cleaning patch and promoted https://review.openstack.org/#/c/337056 instead | 13:57 |
| *** annegentle has joined #openstack-nova | 13:58 | |
| riccardo | hi, i have 5 compute nodes and one controller in place. Is there something I can help you guys? | 13:58 |
| mdbooth | riccardo: This is a channel for nova development. See /topic for user discussion. | 13:59 |
| riccardo | sure, I would like to help with python coding | 14:00 |
| *** luis5tb has joined #openstack-nova | 14:00 | |
| openstackgerrit | Gleb Stepanov proposed openstack/nova: Skip malformed cookies https://review.openstack.org/346090 | 14:00 |
| *** anush has joined #openstack-nova | 14:00 | |
| sfinucan | riccardo: aye, but the focus should be on specific aspects of nova | 14:00 |
| sfinucan | for things like how to access a database there are many better forums | 14:01 |
| riccardo | ok, just let me know how i can help :-) The only constraint: I am a beginner. | 14:01 |
| sfinucan | riccardo: https://wiki.openstack.org/wiki/How_To_Contribute | 14:01 |
| sfinucan | riccardo: bug fixing would be a very good start | 14:01 |
| *** krtaylor has joined #openstack-nova | 14:02 | |
| riccardo | it hasn't to do something with my DB problem. i just wanna contribute anyhow | 14:02 |
| *** ashyoung has joined #openstack-nova | 14:02 | |
| riccardo | bug fix sounds good | 14:02 |
| riccardo | i check the site... | 14:03 |
| bauzas | alaski: around ? | 14:03 |
| bauzas | alaski: back from holiday ? | 14:03 |
| alaski | bauzas: yep | 14:03 |
| bauzas | alaski: so, we have a bug report about updating the instance name https://bugs.launchpad.net/nova/+bug/1620248 | 14:04 |
| openstack | Launchpad bug 1620248 in OpenStack Compute (nova) "Can't rename instance right after creation (regression)" [High,In progress] - Assigned to Sylvain Bauza (sylvain-bauza) | 14:04 |
| bauzas | alaski: if you remember our discussion we had 2 weeks ago... | 14:05 |
| *** thorst has quit IRC | 14:05 | |
| alaski | yeah | 14:05 |
| openstackgerrit | Sylvain Bauza proposed openstack/nova: WIP: Update BuildRequest if instance currently being scheduled https://review.openstack.org/365740 | 14:05 |
| *** thorst has joined #openstack-nova | 14:05 | |
| bauzas | alaski: so, I wonder why we should provide for that... | 14:06 |
| bauzas | alaski: I wrote a quickly patch for modifying then the BuildRequest | 14:06 |
| *** ametts has joined #openstack-nova | 14:06 | |
| bauzas | alaski: https://review.openstack.org/365740 | 14:06 |
| alaski | I'm amazed that anyone is doing that rather than just passing the name in the original request | 14:06 |
| bauzas | alaski: agreed, but I think it could be understanding | 14:07 |
| openstackgerrit | dane-fichter proposed openstack/nova: Use cursive for signature verification https://review.openstack.org/351232 | 14:08 |
| *** marst has quit IRC | 14:08 | |
| bauzas | alaski: so, either we don't accept people updating the instance until it's scheduled, or we need to update the BuildReq | 14:08 |
| alaski | right. I favor the former, but if we can't change that without a microversion there's no point | 14:09 |
| *** amotoki_ has quit IRC | 14:09 | |
| bauzas | alaski: I'm pretty sure some people would like the latter | 14:09 |
| *** hshiina has joined #openstack-nova | 14:10 | |
| alaski | Updating the buildreq is mostly fairly simple. But we have to account for the race where instance creation could be happening as the buildreq is updated. | 14:10 |
| bauzas | alaski: see my change then | 14:10 |
| *** liang_ has joined #openstack-nova | 14:10 | |
| alaski | looking, but I need a sec to digest this | 14:11 |
| mriedem | cdent: we don't need to use the generic-resource-pools bp link in the new commits for bug fixes and unit test adds | 14:11 |
| mriedem | cdent: since we're past FF it's weird to see new patches for a closed bp | 14:11 |
| alaski | bauzas: I think you can simplify this a lot by checking if instance.id is set. If it is the instance exists in a db and instance.save() will work | 14:11 |
| *** ducttape_ has joined #openstack-nova | 14:12 | |
| cdent | mriedem: ah, okay, I was using that to make sure they get linked somewhere in case people start doing a "where's the stuff I'm suppose to look at". but will stop now. | 14:12 |
| *** ekuris has quit IRC | 14:12 | |
| mriedem | cdent: i'd probably just associate them to a bug | 14:13 |
| mriedem | since there should be a bug for the overcommit stuff you and dansmith are trying to hunt down | 14:13 |
| bauzas | alaski: sure, but that will won't work until we delay to persist the instance down in the conductor, right? | 14:13 |
| bauzas | alaski: I mean, here, we already have a created instance | 14:14 |
| *** thorst has quit IRC | 14:14 | |
| alaski | bauzas: currently there are two instances, one from the instance table and buildreq.instance. if .id is set you know you have the one from the instance table | 14:14 |
| *** ababich has quit IRC | 14:14 | |
| *** ducttape_ has quit IRC | 14:15 | |
| bauzas | alaski: right, but I was thinking the build_req.instance was having the id set | 14:15 |
| alaski | you might get buildreq.instance even though there is an instance in the instances table, until instance creation moves to the conductor | 14:15 |
| *** moshele has quit IRC | 14:16 | |
| alaski | bauzas: nope. id is only set if the instance has been written to the db but we persist the buildreq one before doing that | 14:16 |
| *** amotoki has joined #openstack-nova | 14:17 | |
| bauzas | alaski: oh right | 14:17 |
| bauzas | alaski: I just saw that | 14:17 |
| *** michchap has quit IRC | 14:17 | |
| bauzas | I was first thinking about looking at instance._context but lookup at instance.id is better | 14:17 |
| alaski | yeah. .id is the definitive guide to whether or not it's from the instances table | 14:18 |
| *** ducttape_ has joined #openstack-nova | 14:18 | |
| *** ababich has joined #openstack-nova | 14:18 | |
| openstackgerrit | Prateek Arora proposed openstack/nova: Handle ConnectFailure from keystone when neutron is down https://review.openstack.org/327706 | 14:18 |
| openstackgerrit | Christopher Martin proposed openstack/nova: Fix for Nova API Dev Guide https://review.openstack.org/366162 | 14:20 |
| *** mamandle has quit IRC | 14:20 | |
| *** takedakn has joined #openstack-nova | 14:20 | |
| *** gongysh has quit IRC | 14:20 | |
| *** diogogmt has quit IRC | 14:20 | |
| *** links has quit IRC | 14:21 | |
| *** marst has joined #openstack-nova | 14:21 | |
| *** syjulian_ has joined #openstack-nova | 14:21 | |
| *** shangxdy has quit IRC | 14:23 | |
| prateek | mriedem, thanks for the reviewing https://review.openstack.org/#/c/327706/,even though i had requested a review for https://review.openstack.org/#/c/294513/ , in which i had worked on some of your comments and it's been pending for a long time :) | 14:23 |
| *** ametts has quit IRC | 14:24 | |
| mriedem | prateek: i'm in 2 meetings atm, so i'll review what i can review when i can get the time :) | 14:24 |
| *** BjoernT has joined #openstack-nova | 14:25 | |
| *** zigo has quit IRC | 14:25 | |
| prateek | mriedem, sure, thanks ! | 14:25 |
| *** michchap has joined #openstack-nova | 14:26 | |
| *** akshai has quit IRC | 14:27 | |
| *** zz_dimtruck is now known as dimtruck | 14:29 | |
| *** ametts has joined #openstack-nova | 14:29 | |
| *** salv-orlando has joined #openstack-nova | 14:29 | |
| *** prateek has quit IRC | 14:30 | |
| *** felix23ma has quit IRC | 14:30 | |
| *** brian_price has joined #openstack-nova | 14:32 | |
| *** takedakn has quit IRC | 14:32 | |
| *** zigo has joined #openstack-nova | 14:32 | |
| *** salv-orl_ has quit IRC | 14:32 | |
| *** zigo is now known as Guest39147 | 14:33 | |
| *** syjulian_ has quit IRC | 14:33 | |
| mriedem | mdbooth: if you can point me at where functional changes start for the imagebackend series that would save me some time | 14:33 |
| mdbooth | mriedem: Sec | 14:33 |
| *** syjulian_ has joined #openstack-nova | 14:33 | |
| openstackgerrit | Samuel Matzek proposed openstack/nova: Reschedule boot when attach volume fails https://review.openstack.org/246505 | 14:34 |
| mdbooth | Depends how you define functional :) This is the first non-test cleanup: https://review.openstack.org/#/c/333272/ | 14:34 |
| mdbooth | However, its a NFC | 14:34 |
| cdent | sdague, dansmith I'm struggling to replicate the allocation problem seen in temest, locally. And now I have to go to an appt. Will check in afterwards to see what you guys know and what I should do. Can one of you make the bug (if it doesn't already exist) suggested by mriedem a few minutes back. biab. | 14:35 |
| *** jamesdenton has joined #openstack-nova | 14:35 | |
| mdbooth | mriedem: The next one is functional: https://review.openstack.org/#/c/331115/ | 14:35 |
| sdague | cdent: sure, how parallel is your local environment? | 14:35 |
| tojuvone | sdague, dansmith, johnthetubaguy: If some of you could take a glance at this as familiar. I also have POC quite ready against the spec: https://review.openstack.org/310510 | 14:35 |
| mriedem | mdbooth: national football conference? | 14:35 |
| mdbooth | mriedem: Are you suggesting that I continue pushing on the test changes? I'd pretty much abandoned them for now, but I'll happily keep going for a bit. | 14:35 |
| mdbooth | Non Functional Change :) | 14:36 |
| mriedem | mdbooth: at this point with 2 weeks left i'd rather avoid big test refactors | 14:36 |
| mriedem | mdbooth: especially if you're looking at fixing that post-copy network issue | 14:36 |
| mdbooth | mriedem: That's what I thought. I'll leave it. | 14:36 |
| cdent | sdague: not very, which is probably part of the problem, but I'm struggling to come up with a mental model that would explain things, even when parallel. I think my picture of "what's wrong" is incomplete, so mostly I'm just trying to narrow reality a bit. If the bug is able to explain the main issue, that'll probably help. | 14:36 |
| sdague | mdbooth: yeh, typically post freeze we don't merge any non test / non docs code without a bug | 14:36 |
| *** cdent has quit IRC | 14:37 | |
| *** akshai has joined #openstack-nova | 14:37 | |
| *** amotoki has quit IRC | 14:38 | |
| *** jdurgin1 has joined #openstack-nova | 14:38 | |
| *** Guest39147 has quit IRC | 14:39 | |
| sdague | mriedem / dansmith - do we have some reasonable query / interface on allocations to get a debug log dump of what the state of the world looks like? | 14:39 |
| *** zigo_ has joined #openstack-nova | 14:42 | |
| mdbooth | sfinucan: https://bugs.launchpad.net/nova/+bug/1607996 | 14:43 |
| openstack | Launchpad bug 1607996 in OpenStack Compute (nova) "Live migration does not update numa hugepages info in xml" [High,In progress] | 14:43 |
| *** krtaylor has quit IRC | 14:43 | |
| mdbooth | Are you definitely not looking at that any more? | 14:43 |
| mriedem | sdague: idk | 14:44 |
| sfinucan | mdbooth: Not at present, no. I didn't quite get the problem | 14:44 |
| *** claudiub has quit IRC | 14:44 | |
| sfinucan | was he referring to the host XML or guest? | 14:44 |
| sfinucan | *she | 14:44 |
| *** sdake has joined #openstack-nova | 14:44 | |
| mriedem | sdague: we could pull from the DB in a DNM debug patch if needed, directly from the RT i mean | 14:44 |
| mriedem | if necessary | 14:44 |
| mdbooth | sfinucan: Don't know. The bug was/is under discussion in #openstack-meeting-3 | 14:44 |
| *** raj_singh_ has joined #openstack-nova | 14:45 | |
| mriedem | but i don't know what the GET looks like from the placement API | 14:45 |
| mriedem | i thought there was talk on friday that there wasn't a GET | 14:45 |
| sfinucan | mdbooth: under what context? | 14:45 |
| mdbooth | Live migration meeting | 14:45 |
| mdbooth | Ongoing | 14:45 |
| sfinucan | ah | 14:45 |
| sfinucan | afraid there isn't much I can contribute. I can look at it again, but I'd need to ask for more info first | 14:45 |
| openstackgerrit | Gleb Stepanov proposed openstack/python-novaclient: Add evacuate host method https://review.openstack.org/349179 | 14:46 |
| mdbooth | sfinucan: NP | 14:47 |
| *** daemontool_ has quit IRC | 14:47 | |
| *** zigo_ has quit IRC | 14:47 | |
| *** rrecio_ has joined #openstack-nova | 14:47 | |
| *** daemontool has joined #openstack-nova | 14:48 | |
| *** zigo_ has joined #openstack-nova | 14:48 | |
| mriedem | rpodolyaka: sdague: jaypipes: i'm going to -2 the separate placement API DB change for newton https://review.openstack.org/#/c/362766/ - we're past FF and since we aren't testing this i don't think it would be good to land it | 14:52 |
| *** mamandle has joined #openstack-nova | 14:52 | |
| *** annegentle has quit IRC | 14:53 | |
| sdague | mriedem: +1 | 14:53 |
| mriedem | mdbooth: probably a good one for you https://bugs.launchpad.net/nova/+bug/1620341 | 14:53 |
| openstack | Launchpad bug 1620341 in OpenStack Compute (nova) "Removing unused base images removes backing files of active instances" [Undecided,New] | 14:53 |
| *** prateek has joined #openstack-nova | 14:53 | |
| mdbooth | Title sounds fun | 14:54 |
| * mdbooth reads | 14:54 | |
| *** raj_singh_ has quit IRC | 14:54 | |
| mdbooth | mriedem: Paraphrase: image cache manager is broken | 14:54 |
| mdbooth | That's about the level of detail there | 14:55 |
| mriedem | +NFS | 14:55 |
| mriedem | but yeah | 14:55 |
| * mdbooth has a quick look at ICM's shared storage handling | 14:55 | |
| mdbooth | I'm pretty sure it stores a file containing every host which accesses the shared store | 14:56 |
| mdbooth | And filters based on all of them | 14:56 |
| mdbooth | But IIRC there's nothing to stop cleanup running concurrently on different hosts. | 14:56 |
| *** tbachman has quit IRC | 14:57 | |
| kashyap | mdbooth: Randomly chiming in, there are multiple (2 IIRC) config variables related to removing unused base image files. You plan to remove / rework some of them as part of your larger work on Image Cache? | 14:59 |
| *** _ducttape_ has joined #openstack-nova | 14:59 | |
| mdbooth | kashyap: I have no larger work on Image Cache! | 14:59 |
| * kashyap stops making mdbooth responsible for imaginary stuff | 14:59 | |
| mdbooth | I occasionally fix bits of it when I'm compelled to read enough of it to understand some of it. | 14:59 |
| *** pradk has quit IRC | 15:00 | |
| kashyap | Noted :-) | 15:00 |
| *** mjura has quit IRC | 15:01 | |
| *** ashyoung has quit IRC | 15:01 | |
| openstackgerrit | Sean Dague proposed openstack/nova: WIP: additional logging to debug allocations https://review.openstack.org/366186 | 15:02 |
| sdague | mriedem: it will be interesting to see what the run logs for that look like | 15:02 |
| *** rodrigods has quit IRC | 15:02 | |
| *** rodrigods has joined #openstack-nova | 15:02 | |
| *** ducttape_ has quit IRC | 15:02 | |
| markus_z | PaulMurray: apologies for not responding in the live migrate meeting, I was in another meeting too and didn't see it. | 15:03 |
| PaulMurray | markus_z, np | 15:04 |
| PaulMurray | markus_z, does anyone look after the lm bugs ? | 15:04 |
| * PaulMurray expects I should be doing it | 15:04 | |
| *** edand has quit IRC | 15:05 | |
| markus_z | PaulMurray: I'm not aware of anyone, tbh. | 15:05 |
| markus_z | PaulMurray: I keep them alive and wait for my turn for reviews | 15:05 |
| PaulMurray | I will go through them in the next couple of days | 15:05 |
| *** rmart04_ has joined #openstack-nova | 15:06 | |
| markus_z | PaulMurray: One was a regression. The other one an older issue. | 15:06 |
| *** syjulian_ has quit IRC | 15:06 | |
| markus_z | PaulMurray: thanks | 15:06 |
| openstackgerrit | Matt Riedemann proposed openstack/nova: libvirt: improve logging for shared storage check https://review.openstack.org/365140 | 15:06 |
| openstackgerrit | Matt Riedemann proposed openstack/nova: nfs fix for xenial images https://review.openstack.org/329466 | 15:06 |
| rpodolyaka | mriedem: oh, I missed the fact the original change was reverted while I was on vacation :P | 15:06 |
| rpodolyaka | it's a bit disappointing of course, but yeah, I agree it may be too late now | 15:06 |
| *** tbachman has joined #openstack-nova | 15:08 | |
| *** annegentle has joined #openstack-nova | 15:08 | |
| openstackgerrit | Sylvain Bauza proposed openstack/nova: WIP: Update BuildRequest if instance currently being scheduled https://review.openstack.org/365740 | 15:08 |
| bauzas | alaski: just updated based on your comments | 15:08 |
| bauzas | alaski: I think we still need to have lots of conditionals for making sure we don't forget a possible race condition | 15:09 |
| *** andreas_s has quit IRC | 15:09 | |
| *** gouthamr has joined #openstack-nova | 15:10 | |
| *** amotoki has joined #openstack-nova | 15:11 | |
| markus_z | dansmith: FYI: fixes nova-manage privilege escalation: https://review.openstack.org/#/c/352866/ | 15:11 |
| *** thorst has joined #openstack-nova | 15:11 | |
| markus_z | maybe you could have a look at it | 15:11 |
| *** diogogmt has joined #openstack-nova | 15:11 | |
| alaski | bauzas: I don't think you can avoid the race condition without a lock. And I think a first patch should fix the simple issue of updating buildreq.instance and then deal with races in a second patch. | 15:12 |
| *** gouthamr_ has joined #openstack-nova | 15:12 | |
| *** sayalilunkad has quit IRC | 15:13 | |
| *** sayalilunkad has joined #openstack-nova | 15:13 | |
| *** hemnafk has quit IRC | 15:13 | |
| bauzas | alaski: well... | 15:14 |
| *** yamahata has joined #openstack-nova | 15:14 | |
| *** PaulMurray has quit IRC | 15:15 | |
| *** gouthamr has quit IRC | 15:15 | |
| bauzas | alaski: you mean, just amending the BuildReq without really checking whether we should rather use the cellsv2 contextmanager? | 15:16 |
| bauzas | alaski: I thought it was better to add it too, but okaty | 15:16 |
| alaski | bauzas: I mean first solve the issue of not being able to update the buildreq at all. check for instance.id: if it exists instance.update; instance.save, it it doesn't exist pull the buildreq and update it. | 15:17 |
| alaski | then worry about the small race | 15:17 |
| *** nkrinner is now known as nkrinner_afk | 15:18 | |
| *** hemna has joined #openstack-nova | 15:18 | |
| *** thorst has quit IRC | 15:19 | |
| *** pradk has joined #openstack-nova | 15:20 | |
| *** zigo_ is now known as zigo | 15:20 | |
| *** fragatina has joined #openstack-nova | 15:20 | |
| *** mamandle has quit IRC | 15:24 | |
| *** cdent has joined #openstack-nova | 15:26 | |
| *** EinstCrazy has quit IRC | 15:26 | |
| *** fragatina has quit IRC | 15:27 | |
| *** gokrokve has joined #openstack-nova | 15:27 | |
| *** mriedem1 has joined #openstack-nova | 15:27 | |
| *** armax has joined #openstack-nova | 15:29 | |
| *** mriedem has quit IRC | 15:30 | |
| *** thorst has joined #openstack-nova | 15:30 | |
| *** whenry has joined #openstack-nova | 15:31 | |
| *** tbachman has quit IRC | 15:31 | |
| *** sudipto has quit IRC | 15:31 | |
| *** rmart04_ has quit IRC | 15:33 | |
| *** mingyu has quit IRC | 15:34 | |
| sdague | dansmith: I think I figured out the vcpu issue | 15:34 |
| dansmith | yeah? | 15:34 |
| sdague | math is hard | 15:35 |
| *** Jeffrey4l has quit IRC | 15:35 | |
| dansmith | in my local db we are reporting the proper allocation ratio, so that kinda shoots my theory | 15:35 |
| dansmith | unless we're not factoring it into the actual capacity calculation | 15:36 |
| openstackgerrit | Sean Dague proposed openstack/nova: WIP: calculate capacity correctly https://review.openstack.org/366214 | 15:36 |
| sdague | dansmith: I think that's the fix | 15:36 |
| *** rmart04 has joined #openstack-nova | 15:36 | |
| dansmith | hmm, why? | 15:36 |
| dansmith | that would mean we're effectively scaling the reserve right? | 15:37 |
| *** PaulMurray has joined #openstack-nova | 15:37 | |
| dansmith | i.e. we reserve only that amount from the scaled amount | 15:37 |
| *** paul-carlton has quit IRC | 15:37 | |
| *** tonytan4ever has joined #openstack-nova | 15:38 | |
| sdague | ok, well what do we expect total, reserved, allocation_ratio to be there? | 15:38 |
| openstackgerrit | Prateek Arora proposed openstack/nova: Handle ConnectFailure from keystone when neutron is down https://review.openstack.org/327706 | 15:38 |
| *** tonytan_brb has joined #openstack-nova | 15:39 | |
| dansmith | reserved is often zero or small | 15:39 |
| danpb | so consider total capacity is 8 GB and we want 1 GB reserved and alloc ratio is 2 - with old logic we'd report 14 GB, but with fixed logic we're report 15 GB | 15:39 |
| danpb | the new logic sdague suggests does make more sense to me | 15:39 |
| sdague | ok, well it's definitely capacity where we go funky | 15:39 |
| *** andymaier has quit IRC | 15:39 | |
| sdague | let me get some more debug in there to figure that out | 15:39 |
| dansmith | but we should be 4*16 in the jenkins workers | 15:39 |
| dansmith | and I doubt we're starting that many instances in parallel right? | 15:40 |
| bauzas | are you guys talking about how we reserve space in the RT ? | 15:40 |
| *** tblakes has quit IRC | 15:40 | |
| bauzas | that's just a silly math | 15:40 |
| danpb | of course our reserved memory calcs all have a level of crapiness about them | 15:40 |
| sdague | dansmith: we should be 4 * 16 | 15:40 |
| sdague | however, any time we go over 8 we seem to fail | 15:40 |
| sdague | on the capacity check | 15:40 |
| dansmith | sdague: reserved for vcpus is zero right? | 15:40 |
| dansmith | it is on my machine | 15:41 |
| dansmith | so I don't see why your math makes any difference | 15:41 |
| *** pcaruana has joined #openstack-nova | 15:42 | |
| *** rmart04 has quit IRC | 15:42 | |
| *** tonytan4ever has quit IRC | 15:42 | |
| openstackgerrit | Sivasathurappan Radhakrishnan proposed openstack/nova: Remove context object in oslo.log method https://review.openstack.org/317109 | 15:43 |
| sdague | dansmith: ok, let me figure out what's really going on here | 15:43 |
| sdague | I'm adding some more debug there | 15:43 |
| *** sdake has quit IRC | 15:44 | |
| dansmith | okay | 15:44 |
| bauzas | dansmith: sdague: just in case you haven't seen that yet, we just reserve a bit of space by this https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L886 | 15:45 |
| bauzas | dansmith: sdague: then we look at the free ram https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L888 | 15:45 |
| prateek | cdent, sahid , can you please take a look when you have some free time ? https://review.openstack.org/#/c/327706/ | 15:46 |
| bauzas | dansmith: sdague: and then we sum up the ram size for each instance https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L699 | 15:46 |
| sdague | bauzas: right, so this actually won't make any difference on the cpu side | 15:47 |
| sdague | as dansmith correctly states | 15:47 |
| sdague | because the reserved is 0 | 15:47 |
| prateek | jaypipes, can you please take a look when you have some time ? https://review.openstack.org/#/c/294513/ | 15:48 |
| mdbooth | Which release was 13.0.0? | 15:48 |
| mdbooth | Was that liberty? | 15:48 |
| mdbooth | Ah, no. That was mitaka, I think? | 15:49 |
| _gryf | 13 is mitaka, liberty was 12.x | 15:49 |
| mdbooth | Thanks | 15:49 |
| _gryf | mdbooth, https://releases.openstack.org/mitaka/ | 15:50 |
| *** tonytan_brb has quit IRC | 15:51 | |
| bauzas | sdague: correct | 15:51 |
| *** paul-carlton2 has quit IRC | 15:52 | |
| sahid | prateek: done | 15:52 |
| *** tbachman has joined #openstack-nova | 15:52 | |
| *** mriedem1 is now known as mriedem | 15:53 | |
| *** tonytan4ever has joined #openstack-nova | 15:53 | |
| cdent | sdague, dansmith I think we've yet to identify or articulate the actual problem in a complete fashion, and until we can generate a minimal test case that aggrevates the problem we're going to be throwing spitballs that don't stick | 15:54 |
| *** matrohon has quit IRC | 15:55 | |
| *** unicell1 has quit IRC | 15:55 | |
| sdague | cdent: well, some more logging would help, it's exploding in _check_capacity_exceeded | 15:57 |
| sdague | and it really shouldn't be, given what is available at the time | 15:57 |
| cdent | so the questions become: can we construct a minimal test case? Can we do it in a way that it can be run indepdendently of tempest? | 15:57 |
| *** _ducttape_ has quit IRC | 15:58 | |
| *** ducttape_ has joined #openstack-nova | 15:59 | |
| mriedem | you could potentially create a functional test | 15:59 |
| mriedem | but it's a fake virt driver | 15:59 |
| mriedem | so fake RT inventory | 15:59 |
| *** claudiub has joined #openstack-nova | 16:00 | |
| sdague | right, but test for what? We have a real world behavior happening here that we don't yet understand even the bad math that is happening | 16:00 |
| cdent | sdague: basically so we can inspect more quickly. the iteration cycle here is painful. If the answer is "no we can't" that's okay, but I've still yet to read a concise description of the problem so I'm not able to believe "no we can't", yet. | 16:01 |
| *** nic has joined #openstack-nova | 16:01 | |
| *** mgoddard_ has joined #openstack-nova | 16:02 | |
| sdague | 2016-09-06 12:01:16.928 WARNING nova.objects.resource_provider [req-9c93fbd0-ab72-47ad-9479-1132cb85f470 placement service] Attempting to allocate 1 for VCPU. Currently using 1024, amount available 64.0 | 16:02 |
| sdague | that is the crux of the problem | 16:02 |
| *** unicell has joined #openstack-nova | 16:02 | |
| dansmith | sdague: capacity says 1024 are in use and 64 are available for cpus? | 16:03 |
| cdent | If we're driving on guesses at the moment I'd guess that the join or the group by in the sql in the exceed function is wonky | 16:03 |
| sdague | dansmith: yes | 16:03 |
| *** luis5tb has quit IRC | 16:03 | |
| dansmith | 1024/16 = 64 which makes no sense for that worker | 16:03 |
| sdague | correct | 16:03 |
| sdague | 1024 is a crazy value | 16:03 |
| dansmith | that's weird. | 16:03 |
| sdague | this is my nuc | 16:03 |
| dansmith | oh okay, but still | 16:04 |
| sdague | yeh, it's 4 core | 16:04 |
| sdague | and, I know that's not actual usage | 16:04 |
| * cdent inspects the sql | 16:04 | |
| sdague | because I've got other debug logs which are dumping the usage | 16:04 |
| sdague | 2016-09-06 12:01:16.904 DEBUG nova.api.openstack.placement.handlers.allocation [req-9c93fbd0-ab72-47ad-9479-1132cb85f470 placement service] Current Usages for 62b97cd4-5dc8-45d9-89ad-988273895635: Usage(resource_class='VCPU',usage=16), Usage(resource_class='MEMORY_MB',usage=1024), Usage(resource_class='DISK_GB',usage=0), | 16:04 |
| *** TravT_ has joined #openstack-nova | 16:04 | |
| *** gokrokve has quit IRC | 16:05 | |
| sdague | which means for this resource check, it is using the memory value for cpu | 16:05 |
| dansmith | so you booted a guest with 16 vcpus yes? | 16:05 |
| openstackgerrit | Christopher Martin proposed openstack/nova: Fix Boot From Volume size error https://review.openstack.org/364337 | 16:05 |
| *** mgoddard has quit IRC | 16:05 | |
| *** david-lyle_ has joined #openstack-nova | 16:05 | |
| *** david-lyle_ has quit IRC | 16:05 | |
| *** Swami has joined #openstack-nova | 16:06 | |
| sdague | dansmith: nope | 16:06 |
| *** hshiina has quit IRC | 16:06 | |
| sdague | that's complete usages | 16:06 |
| dansmith | sdague: an allocation is one instance | 16:06 |
| sdague | dansmith: it's not an allocation | 16:06 |
| dansmith | oh you're logging usage from the allocation handler I see | 16:06 |
| sdague | https://review.openstack.org/#/c/366186/1/nova/api/openstack/placement/handlers/allocation.py | 16:07 |
| sdague | yeh | 16:07 |
| dansmith | gotcha | 16:07 |
| dansmith | yeah, kinda seems like it's confusing the columns in the sql query or something | 16:07 |
| *** TravT has quit IRC | 16:08 | |
| *** TravT has joined #openstack-nova | 16:10 | |
| openstackgerrit | Matthew Booth proposed openstack/nova: imagecache: Hold shared storage registry lock while running https://review.openstack.org/366239 | 16:10 |
| mdbooth | mriedem: ^^^ But I think the real fix is https://review.openstack.org/#/c/185549/ in mitaka | 16:11 |
| *** paul-carlton2 has joined #openstack-nova | 16:11 | |
| sdague | I'm going to stick another bit of debug on the records to see if there anything useful there | 16:12 |
| *** TravT_ has quit IRC | 16:12 | |
| mriedem | mdbooth: what do you mean by the real fix is in mitaka? | 16:13 |
| mdbooth | mriedem: The reporter mentions they hit the problem in several release, the last of which was liberty | 16:13 |
| mdbooth | The above fix was first in mitaka, and the backport reject for liberty | 16:13 |
| mdbooth | By you :) | 16:13 |
| sdague | ok... so this is weird | 16:13 |
| mdbooth | And I suspect it would resolve their problem | 16:14 |
| mriedem | mdbooth: yeah b/c it required a grenade change for rootwrap filters | 16:14 |
| mdbooth | mriedem: Right, but I thought it was amusing anyway :) | 16:14 |
| *** derekh is now known as derekh_afk | 16:14 | |
| * cdent is increasingly feeling like this sql is not right | 16:14 | |
| sdague | http://paste.openstack.org/show/567204/ | 16:15 |
| *** lpetrut has quit IRC | 16:15 | |
| sdague | we seem to have a joined in a way that we get usage for every resource class type crossed against every resource type | 16:15 |
| *** riccardo has quit IRC | 16:15 | |
| *** lpetrut has joined #openstack-nova | 16:15 | |
| *** rlrossit has joined #openstack-nova | 16:16 | |
| mdbooth | mriedem: Anyway, that code still needs a lock. It came up here: https://review.openstack.org/#/c/334956/ | 16:16 |
| cdent | feh | 16:16 |
| sdague | field 4 is the resource class | 16:16 |
| mdbooth | The code gives the impression it might run concurrently, which raised flags with jaypipes | 16:16 |
| sdague | let me push so I can gitlink things | 16:16 |
| mdbooth | When in fact, that would certainly break. | 16:16 |
| openstackgerrit | Sean Dague proposed openstack/nova: WIP: more and more and more debug https://review.openstack.org/366241 | 16:16 |
| cdent | sdague: i've repeated what you're seeing. nice work. I think that points the finger of doom pretty strongly at the sql | 16:19 |
| sdague | https://github.com/openstack/nova/blob/f9103f6a93b2dcc15e613ee54235fdf54d60027f/nova/objects/resource_provider.py#L705 - so I'm assuming that we should only see 3 records there | 16:20 |
| *** lpetrut has quit IRC | 16:20 | |
| *** gouthamr_ has quit IRC | 16:20 | |
| *** thorst has quit IRC | 16:21 | |
| *** daemontool has quit IRC | 16:21 | |
| *** thorst has joined #openstack-nova | 16:21 | |
| cdent | that's what I think, yeah | 16:21 |
| sdague | ok, so minimally we should probably update the code to explode if we find duplicate keys for our map I would think | 16:22 |
| *** yamahata has quit IRC | 16:23 | |
| *** raunak has joined #openstack-nova | 16:24 | |
| *** rlrossit has quit IRC | 16:25 | |
| *** daemontool has joined #openstack-nova | 16:25 | |
| *** raunak has quit IRC | 16:25 | |
| *** akshai has quit IRC | 16:26 | |
| *** akshai has joined #openstack-nova | 16:29 | |
| *** tonytan_brb has joined #openstack-nova | 16:29 | |
| *** daemontool has quit IRC | 16:29 | |
| *** Apoorva has joined #openstack-nova | 16:29 | |
| *** Sukhdev has joined #openstack-nova | 16:30 | |
| sdague | yeh, it's the left outer join that's the issue | 16:30 |
| *** thorst has quit IRC | 16:30 | |
| sdague | that's why we get usage across all the records, and all the records duplicated | 16:30 |
| sdague | we really want to do a real join with inventory.resource_class_id == allocations.resource_class_id | 16:31 |
| *** stevemar has quit IRC | 16:31 | |
| *** tonytan4ever has quit IRC | 16:32 | |
| *** gouthamr has joined #openstack-nova | 16:33 | |
| sdague | ok, I might have that | 16:33 |
| sdague | I at least get sane allocation records now | 16:33 |
| avolkov | sdague: hi, from https://review.openstack.org/#/c/365015/, do we have some stat about http responses? | 16:33 |
| openstackgerrit | Sean Dague proposed openstack/nova: WIP: correctly join the usage to inventory https://review.openstack.org/366245 | 16:34 |
| sdague | cdent / dansmith ^^^ how does that look to you? | 16:34 |
| cdent | sdague: one sec | 16:34 |
| dansmith | I wouldn't know | 16:34 |
| dansmith | I'll believe it if the logs look righter :) | 16:34 |
| sdague | well, I get sane records now | 16:35 |
| sdague | only 3 of them | 16:35 |
| *** mingyu has joined #openstack-nova | 16:35 | |
| sdague | instead of 9 | 16:35 |
| sdague | and the usage lines up with the resource class I expect | 16:35 |
| *** derekh_afk is now known as derekh | 16:35 | |
| *** raunak has joined #openstack-nova | 16:35 | |
| cdent | sdague: that's the exact same thing that I came up with (while I was one seccing) so I guess that's two votes | 16:35 |
| cdent | just confirming with some runs | 16:36 |
| *** zenoway has quit IRC | 16:36 | |
| *** READ10 has joined #openstack-nova | 16:36 | |
| *** zenoway has joined #openstack-nova | 16:36 | |
| dansmith | sdague: should be able to prove that with a functional test that breaks on the current code right? | 16:36 |
| sdague | dansmith: maybe | 16:36 |
| sdague | so the thing is, I expect this comes and goes because the join columns have no guarunteed order | 16:37 |
| *** tonytan_brb is now known as tonytan4ever | 16:37 | |
| sdague | sorry join rows | 16:37 |
| openstackgerrit | Sivasathurappan Radhakrishnan proposed openstack/nova: Using get() method to prevent KeyError https://review.openstack.org/359424 | 16:38 |
| cdent | sdague: I'm wondering what's missing from the existing allocation violation tests. probably not enough resoruces clases | 16:38 |
| sdague | cdent: enough range of values? | 16:38 |
| cdent | well: not enough classes of resources in the tests such that the cross product happens | 16:38 |
| cdent | although I thought I had one with at least two resource classes in the mix | 16:39 |
| sdague | right, but the issue would only arise if you exceeded one of them | 16:39 |
| sdague | and you managed to get things returned in a way that you were comparing it to the other | 16:39 |
| *** mingyu has quit IRC | 16:40 | |
| cdent | I guess it wasn't quite good enough: https://review.openstack.org/#/c/366144/ | 16:40 |
| cdent | anyway, my local tests confirm your join changes | 16:40 |
| cdent | 3 records with the right data | 16:40 |
| *** zenoway has quit IRC | 16:41 | |
| mdbooth | sdague: That looks correct to me also.. | 16:41 |
| *** sahid has quit IRC | 16:41 | |
| mdbooth | That primary join is essentially joining allocation to inventory | 16:41 |
| *** READ10 has quit IRC | 16:41 | |
| mdbooth | Using only resource_provider in the where clause | 16:41 |
| mdbooth | But there are multiple inventories per resource provider | 16:41 |
| sdague | so, we probably just need a safety check that record length is not greater than len(res_classes) * len(provider_uuids) | 16:42 |
| *** xyang1 has joined #openstack-nova | 16:42 | |
| sdague | that would at least have blown up early on the sql | 16:42 |
| *** fragatina has joined #openstack-nova | 16:42 | |
| mdbooth | resource_provider/resource_class uniquely describes an inventory | 16:42 |
| cdent | if we have functional tests that confirm the sql is doing what it says it should, then having an assertion in the code path seems redundant to me: we need to be able to rely on the sql. If we can't, we need to fix it. | 16:43 |
| sdague | mdbooth: ok, sure, so we do an 'in' check before adding the usage map | 16:43 |
| cdent | the problem in this case is we didnt' have good tests for the sql | 16:43 |
| cdent | s/good/good enough/ | 16:43 |
| mdbooth | sdague: Agreeing with you completely, just showing my working :) | 16:43 |
| sdague | a safety measure is still good | 16:43 |
| *** fragatina has quit IRC | 16:43 | |
| *** lucasagomes is now known as lucas-dinner | 16:44 | |
| *** fragatina has joined #openstack-nova | 16:44 | |
| sdague | ok, I need to get lunch, then I'll go clean this up and rebase to master | 16:44 |
| * cdent tries to think up a reasonable functional test | 16:45 | |
| *** markus_z has quit IRC | 16:46 | |
| *** roeyc has quit IRC | 16:47 | |
| *** pradk has quit IRC | 16:47 | |
| mdbooth | cdent: A resource provider with 2 inventories, with allocations against both. | 16:48 |
| *** jamesdenton has quit IRC | 16:48 | |
| cdent | mdbooth: yeah, I thought I had that, but it appears not | 16:48 |
| cdent | or rather I have it, but I don't have a failure against it | 16:49 |
| * mdbooth is just looking at what the duplicate entries in 'records' result in | 16:49 | |
| *** rlrossit has joined #openstack-nova | 16:49 | |
| *** gszasz has quit IRC | 16:49 | |
| * cdent nods | 16:50 | |
| *** lpetrut has joined #openstack-nova | 16:52 | |
| *** fragatina has quit IRC | 16:52 | |
| mdbooth | This is pretty dense :) | 16:52 |
| *** Sukhdev has quit IRC | 16:53 | |
| *** rama_y has joined #openstack-nova | 16:54 | |
| mdbooth | cdent: So, primary_join is a cartesian join of usage (correct) against all inventories in a single rp | 16:55 |
| *** ihrachys has quit IRC | 16:55 | |
| mdbooth | records, therefore contains a mix of joins, where all but one of the inventories are incorrect | 16:55 |
| openstackgerrit | dane-fichter proposed openstack/nova: Use cursive for signature verification https://review.openstack.org/351232 | 16:56 |
| mdbooth | So the 'last' record may or may not be the correct one | 16:56 |
| sdague | mdbooth: yeh, it's super dense. Which is why I'm in favor of inline safety measures because we've proven that no one can eyeball review it correctly :) | 16:56 |
| *** sdake has joined #openstack-nova | 16:57 | |
| mdbooth | sdague: Sounds like a good idea :) | 16:57 |
| * cdent has a failing test | 16:58 | |
| mdbooth | cdent: provs_with_inv contains all relevant rps | 16:58 |
| cdent | (in the good way) | 16:58 |
| mdbooth | usage_map contains only the last record | 16:58 |
| sdague | heh, you know, jay's comment of "this is this sql" is the right sql he wanted | 16:58 |
| mdbooth | cdent: Now to ensure it's deterministicly failing :) | 16:58 |
| sdague | just missed a join | 16:58 |
| *** hogepodge has quit IRC | 16:58 | |
| *** lpetrut has quit IRC | 16:59 | |
| cdent | sdague: heh, yeah | 16:59 |
| mdbooth | cdent: I assume your test involves executing the sql, right? | 16:59 |
| mdbooth | I wonder what assurances sqlite makes about returned row order in this case | 17:00 |
| *** syjulian_ has joined #openstack-nova | 17:00 | |
| mdbooth | Probably not many :) | 17:00 |
| *** anush has quit IRC | 17:00 | |
| mdbooth | It's a pretty funky query | 17:00 |
| cdent | mdbooth: yeah, this is a functional test that is calling the outer layer object methods, not a unit test on the sql methods | 17:00 |
| mdbooth | cdent: If you're wondering what I'm going on about, btw, it's this: | 17:02 |
| mdbooth | for record in records: | 17:02 |
| mdbooth | usage_map[(record['uuid'], record['resource_class_id'])] = record | 17:02 |
| mdbooth | That will result in usage_map being correct or incorrect depending on whether the last returned row was the correct one or now | 17:03 |
| mdbooth | not | 17:03 |
| *** unicell has quit IRC | 17:03 | |
| *** mgoddard_ has quit IRC | 17:03 | |
| *** hogepodge has joined #openstack-nova | 17:03 | |
| *** mgoddard has joined #openstack-nova | 17:03 | |
| cdent | mdbooth: can you expand on what you're saying there a bit, because, assuming we fix the join as described, why is that still true? | 17:05 |
| mdbooth | cdent: If we fix the join it's not true | 17:05 |
| sdague | do we have a tracking bug yet? | 17:05 |
| cdent | ah, okay, now I get where you're coming from | 17:05 |
| *** bradjones has quit IRC | 17:06 | |
| mdbooth | However, it would make your failing test determined by the returned row order | 17:06 |
| mdbooth | i.e. Your failing test might get false positives with an alternative row order | 17:06 |
| cdent | mdbooth: yeah, but it turns out I made my failing test incorrectly otherwise anyway, so I'm still at the drawing board | 17:06 |
| *** yamahata has joined #openstack-nova | 17:06 | |
| openstackgerrit | Tracy Jones proposed openstack/nova: Fix spelling mistake https://review.openstack.org/366254 | 17:07 |
| cdent | sdague: not that I'm aware of, when I left for my appointment I thought you said you were gonna | 17:07 |
| sdague | oh, I was just digging on the problem, I didn't build the bug yet | 17:07 |
| sdague | I'll do that now | 17:07 |
| *** code-R_ has quit IRC | 17:07 | |
| *** su_zhang has joined #openstack-nova | 17:09 | |
| mdbooth | cdent: Sorry to intrude, I was just being nosy. Enjoy :) | 17:10 |
| *** lpetrut has joined #openstack-nova | 17:10 | |
| * mdbooth has to head home | 17:10 | |
| cdent | mdbooth: not an intrusion at all, thanks for the insight | 17:10 |
| *** armax has quit IRC | 17:11 | |
| *** bradjones has joined #openstack-nova | 17:11 | |
| *** bradjones has quit IRC | 17:11 | |
| *** bradjones has joined #openstack-nova | 17:11 | |
| *** sdake_ has joined #openstack-nova | 17:12 | |
| *** paul-carlton2 has quit IRC | 17:13 | |
| *** syjulian_ has quit IRC | 17:14 | |
| *** sdake has quit IRC | 17:15 | |
| *** pradk has joined #openstack-nova | 17:16 | |
| openstackgerrit | Sean Dague proposed openstack/nova: correctly join the usage to inventory for capacity accounting https://review.openstack.org/366245 | 17:20 |
| sdague | cdent: so, I think if you *actually* want to unit test that, you are going to need to split the _check_capacity_exceeded function in 2 | 17:21 |
| sdague | with the first half being just the SQL | 17:21 |
| sdague | and basically ending at where records are returned | 17:21 |
| cdent | sdague: I think a functional test is more useful in this case, or at least, for me, I'm able to exercise the problem more effectively and realistically (and create a better understanding of what's going) with a functional test | 17:22 |
| sdague | cdent: sure, that's fine too | 17:22 |
| sdague | I do think the key in the bug is complicated sql is complicated, and we got back an unexpected row structure | 17:22 |
| sdague | and long term, isolating that so we know we always get back expected row structure would be nice | 17:23 |
| sdague | anyway, https://review.openstack.org/366245 I think is in the state we want to merge, and it's rebased to master so doesn't depend on anything else | 17:23 |
| *** stevemar has joined #openstack-nova | 17:26 | |
| *** paul-carlton2 has joined #openstack-nova | 17:26 | |
| *** armax has joined #openstack-nova | 17:26 | |
| *** mriedem1 has joined #openstack-nova | 17:26 | |
| *** thorst has joined #openstack-nova | 17:26 | |
| openstackgerrit | Tracy Jones proposed openstack/nova: Fix spelling mistake in libvirt https://review.openstack.org/366262 | 17:28 |
| *** PaulMurray has quit IRC | 17:28 | |
| *** tjones has joined #openstack-nova | 17:28 | |
| *** mriedem has quit IRC | 17:28 | |
| *** sdake_ has quit IRC | 17:28 | |
| *** rama_y has quit IRC | 17:29 | |
| *** sdake has joined #openstack-nova | 17:29 | |
| *** ababich has quit IRC | 17:29 | |
| *** su_zhang has quit IRC | 17:30 | |
| *** jdurgin1 has quit IRC | 17:30 | |
| *** su_zhang has joined #openstack-nova | 17:30 | |
| *** akshai has quit IRC | 17:30 | |
| *** TravT has quit IRC | 17:31 | |
| *** fragatina has joined #openstack-nova | 17:32 | |
| *** fragatina has quit IRC | 17:32 | |
| *** tonytan4ever has quit IRC | 17:33 | |
| *** TravT has joined #openstack-nova | 17:33 | |
| *** fragatina has joined #openstack-nova | 17:33 | |
| *** edand has joined #openstack-nova | 17:33 | |
| *** jheroux has joined #openstack-nova | 17:34 | |
| *** thorst has quit IRC | 17:35 | |
| sdague | gah, pep8 | 17:36 |
| *** jaosorior has quit IRC | 17:36 | |
| openstackgerrit | Sean Dague proposed openstack/nova: correctly join the usage to inventory for capacity accounting https://review.openstack.org/366245 | 17:38 |
| *** harlowja has joined #openstack-nova | 17:38 | |
| cdent | found another (minor) bug so just reporting it for now but will fix it tomorrow. i'm fried at this point: https://bugs.launchpad.net/nova/+bug/1620748 | 17:38 |
| openstack | Launchpad bug 1620748 in OpenStack Compute (nova) "In placement when an attempt is made to write to missing inventory the error message is ugly" [Undecided,New] - Assigned to Chris Dent (cdent) | 17:38 |
| *** amotoki has quit IRC | 17:39 | |
| sdague | yeh, the int ids definitely make dealing with the raw data a bit odd | 17:40 |
| *** tbachman has quit IRC | 17:40 | |
| sdague | it would be nice if the repr for the objects resolved that | 17:40 |
| cdent | you still got the set problem | 17:40 |
| cdent | I'm going to have to give up for the evening; i've reached stupid. will check in later or catch up tomorrow | 17:42 |
| *** cdent has quit IRC | 17:42 | |
| sdague | ok, no worries | 17:43 |
| *** tonytan4ever has joined #openstack-nova | 17:44 | |
| *** tbachman has joined #openstack-nova | 17:44 | |
| *** lamt has joined #openstack-nova | 17:45 | |
| sdague | yay, my run tempest a lot and leak resources meant I even got to see the warning I put in place | 17:48 |
| sdague | 2016-09-06 13:47:13.766 WARNING nova.objects.resource_provider [req-a3758596-5182-4e9e-b7a2-04462bf27f48 placement service] Over capacity for VCPU on resource provider 62b97cd4-5dc8-45d9-89ad-988273895635. Needed: 1, Used: 64, Capacity: 64.0 | 17:48 |
| sdague | 17:48 | |
| *** oomichi- is now known as oomichi | 17:49 | |
| auggy | mriedem1: Jenkins protested at the merged policy files change, https://review.openstack.org/#/c/363768/ so I'm looking into it now | 17:49 |
| *** sdake has quit IRC | 17:49 | |
| dansmith | sdague: I didn't parse that first statement, but is this in CI or locally? | 17:49 |
| sdague | dansmith: it's locally | 17:50 |
| dansmith | sdague: so using some flavor that could really result in 64 vcpus being allocated? | 17:50 |
| *** paul-carlton2 has quit IRC | 17:50 | |
| sdague | my env is pretty dirty now with lots of tempest fails for other reasons, so I've got a leak | 17:50 |
| dansmith | oh | 17:50 |
| sdague | this is trying to allocate a 1 vcpu guest | 17:50 |
| dansmith | you mean you think you've got leaked allocations that don't corespond to instances? | 17:50 |
| sdague | but nova still thinks I've got 64 allocated | 17:51 |
| sdague | yeh | 17:51 |
| dansmith | so you can look at the allocations and see the instance uuids that they came from | 17:51 |
| auggy | johnthetubaguy: the admin port changes, https://review.openstack.org/#/c/357540/ and https://review.openstack.org/#/c/357726/ are all green now and I removed the WIP, let me know if there's anything else you need help with regarding those | 17:51 |
| sdague | dansmith: I think so, what view do you want? | 17:51 |
| dansmith | sdague: it would be good to have a manage command that will do a consistency check on instances and allocations | 17:51 |
| dansmith | sdague: I don't want a view, I'm just saying if you want to go spot check some things to see if there are allocations for deleted instances, you should be able to do that fairly easily | 17:52 |
| sdague | dansmith: yeh, I just assume that's the case. I was just happy to see that warning message was formatted correctly | 17:52 |
| dansmith | sdague: consumer_id in allocations should be the instance uuid | 17:52 |
| sdague | now that it isn't blowing up all the time randomly | 17:52 |
| *** mriedem1 is now known as mriedem | 17:53 | |
| sdague | yeh, based on timestamps it looks like I was leaking 2 - 8 per tempest run | 17:54 |
| *** tblakes has joined #openstack-nova | 17:54 | |
| sdague | but I wasn't even really caring what was passing / failing | 17:54 |
| *** fragatina has quit IRC | 17:54 | |
| sdague | as I was just trying to generate load | 17:54 |
| mriedem | auggy: unrelated, just recheck it | 17:54 |
| dansmith | I wonder if allocation delete is ever blocked for some wrong math reason? | 17:54 |
| mriedem | that cellsv1 job seems to be failing more often on that same test | 17:54 |
| auggy | mriedem: kk will do, just wanted to make sure it wasn't a real issue | 17:55 |
| *** fragatina has joined #openstack-nova | 17:55 | |
| mriedem | auggy: we shouldn't even be running dsvm jobs on a .gitignore change | 17:55 |
| mriedem | we should update project-config to not run dsvm jobs on a change like that, same as how we don't run dsvm jobs for docs or unit test only changes | 17:55 |
| auggy | mriedem: fair point, it seemed odd to me too but i'm paranoid | 17:55 |
| *** jamesdenton has joined #openstack-nova | 17:55 | |
| auggy | mriedem: should i file a bug or something for that as a reminder to make that change? i can work on that | 17:55 |
| *** sdake has joined #openstack-nova | 17:56 | |
| mriedem | auggy: see https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L1339 | 17:56 |
| mriedem | don't need a bug | 17:56 |
| auggy | kk | 17:56 |
| auggy | i'll make a todo for myself and get that in today | 17:56 |
| sdague | dansmith: I am sure there are other things we should check here about how things skew from reality over time | 17:57 |
| *** akshai has joined #openstack-nova | 17:57 | |
| mriedem | auggy: and probably this one https://github.com/openstack-infra/project-config/blob/master/zuul/layout.yaml#L1999 | 17:57 |
| dansmith | sdague: you mean more things we should sanity check inside that capacity transaction? | 17:57 |
| mriedem | so .git.* files should be skipped, like .gitignore and .gitreview | 17:58 |
| sdague | dansmith: we probably need an audit tool that goes across the API | 17:58 |
| *** lyarwood has quit IRC | 17:58 | |
| sdague | dansmith: I just mean "in general" | 17:58 |
| *** mingyu has joined #openstack-nova | 17:58 | |
| dansmith | sdague: such a thing would have to be able to read deleted instances | 17:58 |
| auggy | mriedem: awesome, thanks! | 17:58 |
| openstackgerrit | Andrey Volkov proposed openstack/nova: WIP Move instance creation to conductor https://review.openstack.org/319379 | 17:58 |
| dansmith | so I'd kinda rather do it from manage so we're not building more stuff that relies on deleted instances through the api | 17:58 |
| openstackgerrit | Dan Smith proposed openstack/nova: [placement] Allow inventory to violate allocations https://review.openstack.org/365068 | 17:59 |
| dansmith | sdague: ^ jaypipes is okay with this and it's pretty critical for correctness | 17:59 |
| dansmith | I just removed some more cruft that alex_xu found | 17:59 |
| sdague | dansmith: ok, I'll hit that if you hit my join fix | 18:00 |
| sdague | dansmith: doesn't a nova manage command get weird once placement is separated? especially once it's another db next cycle some time | 18:01 |
| dansmith | sdague: nova manage can hit the placement api if you want, but the instance and these records are already spread across the api and cell databases | 18:02 |
| *** edand has quit IRC | 18:02 | |
| dansmith | sdague: I'm not sure I'm really able to validate that sql change.. and I thought we were going to wait for a functional test? | 18:03 |
| dansmith | well, maybe I could validate it if I wrote said tests, but.. | 18:03 |
| *** mingyu has quit IRC | 18:04 | |
| sdague | well, the validation at this point is just that KeyError isn't raised | 18:05 |
| sdague | that's where we went wrong, we took 9 rows back from the sql query, squashed them down to 3 rows, kind of randomly in a dictionary, and assumed they were the rows we wanted (they often weren't) | 18:06 |
| *** crinkle_ is now known as crinkle | 18:07 | |
| *** rlrossit_ has joined #openstack-nova | 18:07 | |
| *** tbachman_ has joined #openstack-nova | 18:07 | |
| *** tbachman has quit IRC | 18:08 | |
| *** tbachman_ is now known as tbachman | 18:08 | |
| dansmith | sdague: if I remove that functional change and leave the key assert thing, I don't get any extra fails | 18:09 |
| sdague | dansmith: in a real environment? | 18:09 |
| dansmith | no, in functional tests | 18:09 |
| *** oanson has joined #openstack-nova | 18:10 | |
| *** rlrossit has quit IRC | 18:10 | |
| *** ssurana has joined #openstack-nova | 18:12 | |
| dansmith | so I modified one of our tests to hit that keyerror, but your functional change doesn't make it go away | 18:14 |
| sdague | dansmith: what did you modify? | 18:14 |
| sdague | so, I've tested this live, and it's definitely the real fix. I am sure the functional tests are lacking here | 18:14 |
| dansmith | er, wait | 18:14 |
| dansmith | actually, I'm hitting a different keyerror in the code you added, but not the raise you want | 18:15 |
| auggy | oh wow i just read markus isn't doing bugs or much nova anymore :( | 18:16 |
| auggy | i swear, go on a holiday for a day... | 18:17 |
| *** pradk has quit IRC | 18:18 | |
| *** fragatin_ has joined #openstack-nova | 18:22 | |
| tjones | yeah too bad - he did a great job. it's a hard one | 18:23 |
| mriedem | Swami: reviewed https://review.openstack.org/#/c/275073/ | 18:24 |
| Swami | mriedem: thanks | 18:24 |
| *** fragatina has quit IRC | 18:25 | |
| dansmith | sdague: so you're adding an extra filter on resource_class, but it seems like the check you're doing is to filter out multiple rows of the same class for two resource providers? | 18:25 |
| sdague | dansmith: yeh, it looks like the functional tests are too clever by half and try to do a whole bunch of stuff in a sequence so it's a bit less real. I'll leave someone else to make those sane. That's the real | 18:25 |
| sdague | dansmith: no, the key is a tuple | 18:25 |
| *** tri2sing has joined #openstack-nova | 18:25 | |
| sdague | of rp uuid and resource class | 18:25 |
| dansmith | sdague: I know | 18:26 |
| tjones | mriedem: are we at the point where we are pausing on removing mox for newton ? | 18:26 |
| mriedem | tjones: yes | 18:26 |
| tjones | ok thanks | 18:26 |
| mriedem | there is a thread in the ML on moving those to an ocata blueprint | 18:26 |
| *** oomichi has quit IRC | 18:26 | |
| *** oomichi has joined #openstack-nova | 18:27 | |
| tjones | yeah i missed it but i figured we were there. | 18:27 |
| *** danpb has quit IRC | 18:27 | |
| dansmith | sdague: I modified a test so that one RP has two resource classes, which is realistic and I thought what you were saying was the problem that we get the cross-product of the thing we're looking for, such that we have multiple (uuid, class) pairs in one result set | 18:28 |
| openstackgerrit | Ryan Rossiter proposed openstack/nova: Log trace when updating host states https://review.openstack.org/366283 | 18:29 |
| *** oanson has quit IRC | 18:29 | |
| dansmith | sdague: but I don't hit that trigger if I remove your extra resource_class_id filter from the query | 18:29 |
| *** thorst has joined #openstack-nova | 18:32 | |
| *** jamesdenton has quit IRC | 18:32 | |
| sdague | dansmith: yep, I don't know why the tests don't match the reality. I'll let jaypipes / cdent run with that one | 18:33 |
| dansmith | okay | 18:34 |
| jaypipes | dansmith, sdague: what's up? sorry, been in meetings all day :( | 18:35 |
| dansmith | jaypipes: summarizing on a review, just a sec | 18:35 |
| jaypipes | k | 18:35 |
| *** pradk has joined #openstack-nova | 18:35 | |
| dansmith | jaypipes: https://review.openstack.org/#/c/366245/3 | 18:37 |
| dansmith | jaypipes: sdague figured out that the VCPU thing is due to us not always picking the right inventory result row during the capacity check, but I can't write a functional test to reproduce/prove it which I think is important | 18:37 |
| openstackgerrit | Ryan Rossiter proposed openstack/nova: Warn when using scheduler with debug mode https://review.openstack.org/366289 | 18:37 |
| *** thorst has quit IRC | 18:39 | |
| *** lamt has quit IRC | 18:40 | |
| *** mfedosin has quit IRC | 18:40 | |
| jaypipes | dansmith: k. do you need me to pull the above patch and write a func test for the multiple-providers-multiple-resources case? | 18:41 |
| dansmith | jaypipes: yeah | 18:41 |
| dansmith | jaypipes: write a test that fails without the extra query addition | 18:41 |
| jaypipes | dansmith: you got it. | 18:42 |
| dansmith | jaypipes: that's what I've been trying to do, but I'm not smart enough apparently | 18:42 |
| *** oomichi has quit IRC | 18:42 | |
| jaypipes | dansmith: I've got more meetings but will try my best... | 18:42 |
| dansmith | are you saying you think we need multipiple RP each with multiple classes to hit that? | 18:42 |
| *** baoli has joined #openstack-nova | 18:42 | |
| jaypipes | dansmith: yeah. | 18:42 |
| dansmith | we should only have one RP in the current test tempest runs I think, so I'm not sure how we'd have hit that... | 18:42 |
| *** baoli_ has quit IRC | 18:42 | |
| jaypipes | (I think) :) | 18:43 |
| sdague | yeh, this seemed hittable with only 1 RP | 18:43 |
| * dansmith puzzles | 18:43 | |
| sdague | http://logs.openstack.org/41/366241/1/check/gate-tempest-dsvm-neutron-placement-full-ubuntu-xenial-nv/48a0a01/logs/apache/placement-api.txt.gz#_2016-09-06_16_36_22_242 | 18:44 |
| sdague | that's in a run with a bunch of unmerged logging | 18:44 |
| *** lamt has joined #openstack-nova | 18:44 | |
| *** TravT has quit IRC | 18:45 | |
| sdague | we definitely only have 1 resource provider there | 18:45 |
| dansmith | sdague: what allocation has 192 of anything? | 18:45 |
| sdague | mem | 18:45 |
| dansmith | we're using a 192MB memory flavor in tempest in the gate? | 18:46 |
| sdague | we're using 64MB flavors | 18:46 |
| *** prateek_ has joined #openstack-nova | 18:46 | |
| *** TravT has joined #openstack-nova | 18:46 | |
| sdague | that's the mem usage | 18:46 |
| dansmith | that's what I thought, so no allocation should be more than 64 | 18:46 |
| jaypipes | dansmith: the resource tracker WARN log message right above there shows 64 vCPU available, using 128 vCPU, requesting 1 vCPU. | 18:46 |
| sdague | dansmith: that's *usage* | 18:46 |
| dansmith | sdague: oh the log says "allocation record: ..." | 18:47 |
| sdague | yes | 18:47 |
| sdague | it's the rows from the giant sql monster | 18:47 |
| sdague | it was the best name I had for it | 18:47 |
| *** rama_y has joined #openstack-nova | 18:47 | |
| dansmith | sdague: then that log message should be different :) | 18:47 |
| jaypipes | sdague: that's the correct name/term :) | 18:47 |
| sdague | ok, tell me what the debug message should be :) | 18:47 |
| dansmith | jaypipes: he says "allocation record: $usage" .. that's not right to me :) | 18:47 |
| *** raunak has quit IRC | 18:48 | |
| *** priteau has joined #openstack-nova | 18:48 | |
| dansmith | jaypipes: as in Usage() which is a sum from the resource provider | 18:48 |
| *** oomichi has joined #openstack-nova | 18:48 | |
| sdague | https://review.openstack.org/#/c/366241/1/nova/objects/resource_provider.py@706 | 18:48 |
| jaypipes | dansmith: yes, true. I was just saying "allocation record" is the right term here. | 18:48 |
| dansmith | jaypipes: how is it right? | 18:48 |
| jaypipes | dansmith: doh, no, you are 100% correct, sorry. | 18:49 |
| dansmith | sdague: yeah, I see why you said that, but since we have actual "Allocation" reocrds, I think it's wrong to say that :) | 18:49 |
| *** prateek has quit IRC | 18:49 | |
| sdague | dansmith: ok, tell me a different word, that's the best I had when I was hacking to the bottom of this | 18:49 |
| sdague | anyway, terminology asside | 18:49 |
| dansmith | sdague: capacity or usage or something | 18:49 |
| sdague | you can see the N^2 matrix there | 18:50 |
| *** prateek has joined #openstack-nova | 18:50 | |
| dansmith | sdague: the terminology had me concerned we were trying to allocate a bad value is my point | 18:50 |
| sdague | ok | 18:50 |
| dansmith | sdague: anyway, those are just three records, which we expect.. this is after your fix, right? | 18:51 |
| *** prateek_ has quit IRC | 18:51 | |
| sdague | dansmith: no, that's before the fix | 18:51 |
| dansmith | sdague: okay, but that's not NxM | 18:51 |
| sdague | and there a 9 records | 18:51 |
| dansmith | that's what I would expect to see | 18:51 |
| dansmith | oh, lol.. scroll up | 18:51 |
| sdague | :) | 18:52 |
| dansmith | yeah, so I don't know why I can | 18:52 |
| *** oomichi has quit IRC | 18:52 | |
| dansmith | why I can't repro that | 18:52 |
| *** oomichi has joined #openstack-nova | 18:52 | |
| sdague | yeh, me either | 18:52 |
| dansmith | that plus your fix kindaish makes sense to my SQL-dumb brain, but.. I want to see it in a test and I can't make it do that | 18:52 |
| openstackgerrit | Sylvain Bauza proposed openstack/nova: WIP: Update BuildRequest if instance currently being scheduled https://review.openstack.org/365740 | 18:52 |
| bauzas | alaski: ^ updated for a quickiest patch | 18:53 |
| bauzas | alaski: if you agree with it, I'll add the UTS | 18:53 |
| bauzas | UTs | 18:53 |
| alaski | bauzas: was just commenting on the previous version. I'll finish that and update and look again | 18:54 |
| sdague | ok, well, I'll leave that to jaypipes. Going to walk away from computer for a bit. | 18:54 |
| jaypipes | dansmith: ah, got it... | 18:54 |
| bauzas | alaski: heh, sorry about that :) | 18:55 |
| alaski | no worries, I do it too :) | 18:55 |
| jaypipes | dansmith: TestAllocationListCreateDelete.test_alllocation_list_create() uses multiple resource providers, but only one resource class inventory record for each resource provider. If we change that to be multiple resource classes per provider, it should fail. | 18:55 |
| jaypipes | dansmith: I will update and test pre-patch and post-patch. | 18:56 |
| *** raunak has joined #openstack-nova | 18:59 | |
| dansmith | jaypipes: that is *precisely* what I have done and I dont' see the fail | 18:59 |
| dansmith | jaypipes: http://pastebin.com/cmcvaLqJ | 19:00 |
| dansmith | jaypipes: that case of multiple classes for one RP is definitely a giant gap in our current tests for sure | 19:00 |
| dansmith | especially since that's a realistic scenario | 19:01 |
| *** prateek_ has joined #openstack-nova | 19:02 | |
| *** raunak has quit IRC | 19:02 | |
| jaypipes | dansmith: ok. well, lemme play around with this and see if I can trigger it. | 19:02 |
| dansmith | jaypipes: yep, cool | 19:02 |
| dansmith | in about 30 I have to run off for a bit, but will check in with you this afternoon | 19:02 |
| *** prateek has quit IRC | 19:05 | |
| *** prateek_ has quit IRC | 19:06 | |
| alaski | bauzas: added some comments. It still has some checks I think you can remove, but looks close | 19:07 |
| *** prateek_ has joined #openstack-nova | 19:07 | |
| bauzas | alaski: okay, fair enough, I'll work on that tomorrow with the UTs so we could try to merge it before RC1 | 19:08 |
| openstackgerrit | Ryan Rossiter proposed openstack/nova: Warn when using scheduler with debug mode https://review.openstack.org/366289 | 19:08 |
| bauzas | alaski: that's a regression hence me running for it | 19:08 |
| bauzas | auggy: FWIW, saw your email | 19:08 |
| *** parora has joined #openstack-nova | 19:09 | |
| bauzas | auggy: I think we need to pay close attention to new and in progress bugs to see whether they're related to some regression | 19:09 |
| *** gjayavelu has joined #openstack-nova | 19:09 | |
| alaski | bauzas: yep, we need to get this fixed | 19:09 |
| *** akshai has quit IRC | 19:09 | |
| bauzas | auggy: I tried to look around all the open bugs, but I really need help here :) | 19:09 |
| *** Sukhdev has joined #openstack-nova | 19:09 | |
| openstackgerrit | Ryan Rossiter proposed openstack/nova: Log trace when updating host states https://review.openstack.org/366283 | 19:11 |
| *** pcaruana has quit IRC | 19:11 | |
| *** prateek_ has quit IRC | 19:12 | |
| *** prateek_ has joined #openstack-nova | 19:13 | |
| *** cdent has joined #openstack-nova | 19:14 | |
| *** Guest78236 is now known as melwitt | 19:14 | |
| bauzas | rlrossit_: so, yeah, I agree that running DEBUG for the scheduler is a very bad idea... | 19:14 |
| bauzas | rlrossit_: like drinking while driving | 19:14 |
| bauzas | rlrossit_: but there, should we just add some ads saying that it's bad, or rather find some situations where DEBUG logs are unnecessary? | 19:15 |
| cdent | dansmith: you've made me feel better. I was unable to write a failing functional test too and left for dinner in frustration. I wonder if it is sqlite artifact (or mysql). I can demonstrate the bad behavior in devstack no problem. | 19:15 |
| bauzas | rlrossit_: I tend to like the latter :) | 19:15 |
| auggy | bauzas: lol i'm not a pro either but i will do what i can ;) | 19:15 |
| dansmith | cdent: yeah, I dunno, but this seems like an odd thing to be an artifact | 19:15 |
| bauzas | auggy: bug triaging is one way to come by Nova :) | 19:15 |
| *** cleong has quit IRC | 19:15 | |
| auggy | bauzas: yeah, i've been pretty active with helping markus this past year | 19:16 |
| rlrossit_ | bauzas: I have decided to do both | 19:16 |
| bauzas | rlrossit_: \o/ | 19:16 |
| * auggy has learned a lot from nova bugs | 19:16 | |
| *** mriedem has quit IRC | 19:16 | |
| rlrossit_ | I'm saying debug=True with scheduler is a bad thing, and then I also move the main debug culprit to trace() | 19:16 |
| *** parora has quit IRC | 19:16 | |
| rlrossit_ | there's 2 patches | 19:16 |
| bauzas | rlrossit_: I just saw the second one | 19:16 |
| bauzas | rlrossit_: tbh, I'm very not sold about saying it's a bad thing | 19:17 |
| bauzas | rlrossit_: that's just life | 19:17 |
| cdent | dansmith: that's all I had, going back to not being here | 19:17 |
| *** cdent has quit IRC | 19:17 | |
| rlrossit_ | https://review.openstack.org/366289 is "Hey you shouldn't do this", and https://review.openstack.org/366283 is "well, you did it, let's not make it hurt so much" | 19:17 |
| bauzas | rlrossit_: yeah I very gently -1d the former :D | 19:17 |
| *** fragatin_ has quit IRC | 19:17 | |
| *** fragatina has joined #openstack-nova | 19:18 | |
| bauzas | rlrossit_: so the problem is more that you need DEBUG for getting important info when some bad scheduling happens | 19:19 |
| bauzas | rlrossit_: I mean, some operators want to use DEBUG in production because they think they miss some crucial information about some bad scheduling | 19:19 |
| bauzas | rlrossit_: but just changing the log level doesn't help them IMHO | 19:20 |
| bauzas | rlrossit_: we should rather think about why they need to use DEBUG | 19:20 |
| *** pradk- has joined #openstack-nova | 19:20 | |
| bauzas | rlrossit_: see ? | 19:20 |
| *** pradk has quit IRC | 19:21 | |
| rlrossit_ | yeah, I don't disagree with that | 19:21 |
| *** parora has joined #openstack-nova | 19:21 | |
| rlrossit_ | and granted, when I was doing this performance stuff, I wouldn't go looking in the logs to see that I shouldn't be running with debug on. | 19:21 |
| rlrossit_ | it was slow, and I didn't know why or where until I commented logging out | 19:22 |
| *** pradk- is now known as pradk | 19:22 | |
| *** lamt has quit IRC | 19:24 | |
| *** rmart04 has joined #openstack-nova | 19:24 | |
| bauzas | rlrossit_: that's why we had some BP related to that | 19:24 |
| bauzas | rlrossit_: lemme find it | 19:25 |
| *** tesseract- has quit IRC | 19:25 | |
| rlrossit_ | yeah mriedem linked it to me on Friday | 19:25 |
| *** prateek_ has quit IRC | 19:25 | |
| rlrossit_ | cfriesen is working on it I believe | 19:25 |
| *** lpetrut has quit IRC | 19:25 | |
| bauzas | rlrossit_: the idea was to stack all the filter logs so that where we were having a NoValidHost, then it was popped | 19:25 |
| *** akshai has joined #openstack-nova | 19:25 | |
| rlrossit_ | yup | 19:25 |
| rlrossit_ | "in-memory logging" of sorts | 19:26 |
| bauzas | rlrossit_: well, I haven't seen much of a progress here, so I think you should maybe see how cfriesen and you could resurrect that | 19:26 |
| melwitt | I had thought turning off debug log level for performance testing was a given, be it openstack or not | 19:26 |
| bauzas | the Ocata cycle will open soon and I'd be very happy to see that | 19:26 |
| rlrossit_ | melwitt: well, I didn't think debug logging would be *that* bad | 19:27 |
| bauzas | melwitt: we should rename s/DEBUG/EDONTDOTHISFORPRODUCTION I think :p | 19:27 |
| dansmith | I thought that we actually hear that most big ops actually run with debug on all the time because they have to have it to diagnose any real issue | 19:27 |
| rlrossit_ | bauzas: still doesn't stop some people ;) | 19:27 |
| dansmith | i.e. they pay the penalty for the benefit of being able to fix things | 19:27 |
| bauzas | dansmith: yup, why I pointed out cfriesen's BP | 19:28 |
| dansmith | that may have come via sdague a while ago | 19:28 |
| melwitt | we didn't. debug is definitely not on unless something bad happens and debugging is needed | 19:28 |
| rlrossit_ | dansmith: yeah... that's what we're starting to live with | 19:28 |
| dansmith | melwitt: okay | 19:28 |
| *** diogogmt has quit IRC | 19:29 | |
| *** hemanthm is now known as hemanthm|away | 19:29 | |
| openstackgerrit | Swaminathan Vasudevan proposed openstack/nova: Implement setup_networks_on_host for Neutron networks https://review.openstack.org/275073 | 19:30 |
| bauzas | rlrossit_: grah, can't find cfriesen's BP | 19:30 |
| bauzas | rlrossit_: I'd have checked the whiteboard to see the status | 19:30 |
| *** diogogmt has joined #openstack-nova | 19:31 | |
| *** tbachman_ has joined #openstack-nova | 19:31 | |
| bauzas | grah, can someone tell me how I can find the gerrit username for cfriesen so I could search for his open changes ? | 19:31 |
| *** coolsvap_ has quit IRC | 19:32 | |
| *** tbachman has quit IRC | 19:32 | |
| *** tbachman_ is now known as tbachman | 19:32 | |
| bauzas | nevermind, found it | 19:32 |
| bauzas | rlrossit_: https://review.openstack.org/#/c/331306/ has some implementation concerns, possibly good to disccuss that with the owner | 19:33 |
| *** claudiub has quit IRC | 19:34 | |
| bauzas | rlrossit_: in particular given your testbed of 400 nodes, could be interesting to see the memory usage for that | 19:34 |
| *** Apoorva has quit IRC | 19:34 | |
| alaski | I know that rax runs(ran) with debug logs on, because tracking down issues would have been near impossible without it. But cells may have been (i.e. was) a factor there. | 19:35 |
| *** rmart04 has quit IRC | 19:35 | |
| *** parora has quit IRC | 19:36 | |
| alaski | efforts like cfriesens to only dump logs on failure would be a huge help there | 19:36 |
| bauzas | alaski: yup | 19:36 |
| rlrossit_ | bauzas: I don't see that using much more memory than holding 400 host state maps in memory | 19:36 |
| *** thorst has joined #openstack-nova | 19:36 | |
| bauzas | rlrossit_: well, that's different | 19:36 |
| bauzas | rlrossit_: holding 400 host state dicts is one thing | 19:37 |
| bauzas | rlrossit_: allowing to write in memory long strings is another, I guess | 19:37 |
| bauzas | as you stated, logs can be very verbose :) | 19:37 |
| rlrossit_ | I guess what I'm trying to say is: overall, it's not that much *more* memory | 19:38 |
| rlrossit_ | ram is cheap, file I/O is expensive | 19:38 |
| rlrossit_ | well, that debug that I moved down to trace doesn't really help much in the case of scheduling failures anyways :) | 19:39 |
| bauzas | rlrossit_: but you can define a log factory for each level you know :) | 19:40 |
| *** rcernin has joined #openstack-nova | 19:41 | |
| bauzas | anyway, I just want to tell that merging your patch won't really solve the problem | 19:41 |
| bauzas | it could just reduce the verbosity, sure | 19:41 |
| bauzas | but that would still need some further improvment | 19:41 |
| *** thorst has quit IRC | 19:45 | |
| *** sdake has quit IRC | 19:46 | |
| *** moshele has joined #openstack-nova | 19:48 | |
| *** rcernin has quit IRC | 19:48 | |
| *** pradk has quit IRC | 19:49 | |
| sdague | jaypipes: you have that test fix, because if not, I have it, now that you gave the hint | 19:50 |
| jaypipes | sdague: meetings :( | 19:52 |
| sdague | ok | 19:52 |
| *** TravT has quit IRC | 19:55 | |
| openstackgerrit | Sean Dague proposed openstack/nova: correctly join the usage to inventory for capacity accounting https://review.openstack.org/366245 | 19:56 |
| openstackgerrit | Sean Dague proposed openstack/nova: DNM: test SQL fix https://review.openstack.org/366313 | 19:56 |
| *** eharney has quit IRC | 19:56 | |
| *** devananda is now known as devananda|travel | 19:57 | |
| sdague | ug, that's not really it, I need to futz with this a bit more | 19:59 |
| *** davidlenwell has quit IRC | 19:59 | |
| *** mvk has quit IRC | 20:01 | |
| *** kfarr has joined #openstack-nova | 20:02 | |
| *** raunak has joined #openstack-nova | 20:03 | |
| *** sdake has joined #openstack-nova | 20:03 | |
| *** joshblake has joined #openstack-nova | 20:04 | |
| cfriesen | sfinucan: you on? | 20:06 |
| *** eharney has joined #openstack-nova | 20:08 | |
| openstackgerrit | Sean Dague proposed openstack/nova: correctly join the usage to inventory for capacity accounting https://review.openstack.org/366245 | 20:09 |
| *** raunak has quit IRC | 20:09 | |
| openstackgerrit | Sean Dague proposed openstack/nova: DNM: test SQL fix https://review.openstack.org/366313 | 20:09 |
| *** tbachman has quit IRC | 20:10 | |
| *** tbachman has joined #openstack-nova | 20:11 | |
| sdague | yeh, still doesn't quite trigger it, I'm not sure now | 20:11 |
| *** ljxiash has quit IRC | 20:11 | |
| *** davidlenwell has joined #openstack-nova | 20:12 | |
| *** TravT has joined #openstack-nova | 20:13 | |
| *** edmondsw has quit IRC | 20:13 | |
| bauzas | cfriesen: sfinucan lives in Ireland, so probably off for tonight | 20:15 |
| *** raunak has joined #openstack-nova | 20:18 | |
| *** unicell has joined #openstack-nova | 20:18 | |
| *** kevinz has quit IRC | 20:23 | |
| openstackgerrit | Andrew Laski proposed openstack/nova: Annotate db models that have moved to the nova_api db https://review.openstack.org/366327 | 20:26 |
| *** praneshp has joined #openstack-nova | 20:29 | |
| *** salv-orl_ has joined #openstack-nova | 20:30 | |
| *** moshele has quit IRC | 20:33 | |
| *** salv-orlando has quit IRC | 20:33 | |
| *** thorst has joined #openstack-nova | 20:33 | |
| cfriesen | bauzas: ah, thanks. will try tomorrow | 20:33 |
| bauzas | np | 20:34 |
| bauzas | cfriesen: I had a convo with rlrossit_ about the scheduler logging BP you wrote | 20:35 |
| bauzas | cfriesen: hopefully you'll have time to work on it for Ocata ? | 20:35 |
| *** thorst has quit IRC | 20:39 | |
| openstackgerrit | Swaminathan Vasudevan proposed openstack/nova: Implement setup_networks_on_host for Neutron networks https://review.openstack.org/275073 | 20:40 |
| *** ayoung has quit IRC | 20:41 | |
| *** moshele has joined #openstack-nova | 20:41 | |
| *** pradk has joined #openstack-nova | 20:44 | |
| *** cdelatte has quit IRC | 20:44 | |
| sdague | jaypipes: ok, I think I figured it out | 20:46 |
| sdague | I needed 2 consumers | 20:46 |
| sdague | otherwise during allocations it deleted all the usage for the current consumer | 20:47 |
| openstackgerrit | Sean Dague proposed openstack/nova: DNM: test SQL fix https://review.openstack.org/366313 | 20:47 |
| openstackgerrit | Sean Dague proposed openstack/nova: correctly join the usage to inventory for capacity accounting https://review.openstack.org/366245 | 20:47 |
| jaypipes | sdague: aha. | 20:47 |
| sdague | so 1 resource_provider, > 1 active consumer, > 1 resource class | 20:47 |
| jaypipes | right, that makes sense. | 20:47 |
| sdague | there is a dedicated test for this in the patch now | 20:48 |
| jaypipes | sdague: nice digging. i literally just got off my 11th meeting today (not kidding) | 20:48 |
| sdague | yep, no worries. I learned a bunch about the whole system in the process, so that's always good | 20:48 |
| cfriesen | bauzas: hopefully, yes. we were kind of late out the gate for N...other stuff going on. | 20:49 |
| jaypipes | sdague: thank you very much for your help on all this over last couple weeks. I really appreciate it. | 20:49 |
| sdague | jaypipes: happy to help | 20:49 |
| *** agireud has quit IRC | 20:52 | |
| sdague | I think you owe me a cookie or drink or something in barcelona for debugging your SQL though :) | 20:52 |
| *** rlrossit_ has quit IRC | 20:54 | |
| *** TravT has quit IRC | 20:55 | |
| *** agireud has joined #openstack-nova | 20:59 | |
| *** mriedem has joined #openstack-nova | 21:01 | |
| *** sdake has quit IRC | 21:02 | |
| sdague | jaypipes: it would be cool if you could review - https://review.openstack.org/#/c/366245/ for correctness | 21:03 |
| *** kylek3h has quit IRC | 21:04 | |
| *** derekh has quit IRC | 21:05 | |
| *** raunak has quit IRC | 21:06 | |
| openstackgerrit | Anusha Unnam proposed openstack/nova: List soft_deleted instances https://review.openstack.org/366347 | 21:08 |
| dansmith | sdague: I'm pulling it down to play now | 21:09 |
| sdague | dansmith: you should get the fail on the DNM patch | 21:10 |
| *** mmedvede has quit IRC | 21:11 | |
| *** mmedvede has joined #openstack-nova | 21:11 | |
| *** raunak has joined #openstack-nova | 21:11 | |
| *** ljxiash has joined #openstack-nova | 21:12 | |
| *** raunak has quit IRC | 21:12 | |
| dansmith | sdague: found a bug, fixing locally and will push it up | 21:12 |
| *** mvk has joined #openstack-nova | 21:13 | |
| sdague | cool | 21:13 |
| openstackgerrit | Dan Smith proposed openstack/nova: correctly join the usage to inventory for capacity accounting https://review.openstack.org/366245 | 21:13 |
| sdague | dansmith: you deleted the fix | 21:14 |
| dansmith | hah | 21:14 |
| sdague | https://review.openstack.org/#/c/366245/6..7/nova/objects/resource_provider.py | 21:14 |
| dansmith | because I was testing | 21:14 |
| dansmith | sorry | 21:14 |
| sdague | the map_key bit works | 21:14 |
| sdague | without the explicit str() | 21:14 |
| jaypipes | sdague: +2 from me. | 21:15 |
| *** joshblake has quit IRC | 21:15 | |
| dansmith | sdague: it doesn't here, it fails | 21:15 |
| dansmith | sdague: with string format | 21:16 |
| *** ljxiash has quit IRC | 21:16 | |
| openstackgerrit | Dan Smith proposed openstack/nova: correctly join the usage to inventory for capacity accounting https://review.openstack.org/366245 | 21:16 |
| sdague | odd | 21:16 |
| dansmith | and it should fail of course | 21:16 |
| *** tonytan_brb has joined #openstack-nova | 21:17 | |
| sdague | oh... right | 21:17 |
| dansmith | raise KeyError("%s already in usage_map, bad query" % map_key) | 21:17 |
| dansmith | TypeError: not all arguments converted during string formatting | 21:17 |
| *** raunak has joined #openstack-nova | 21:18 | |
| *** tri2sing has quit IRC | 21:19 | |
| *** tonytan4ever has quit IRC | 21:19 | |
| *** tri2sing has joined #openstack-nova | 21:19 | |
| *** Apoorva has joined #openstack-nova | 21:21 | |
| *** dimtruck is now known as zz_dimtruck | 21:23 | |
| sdague | yep | 21:25 |
| sdague | I guess I had just gone blind in my *stacktest* buffer | 21:26 |
| *** rmart04 has joined #openstack-nova | 21:26 | |
| dansmith | you get a similar enough error that it seems like it's working | 21:27 |
| sdague | yeh | 21:27 |
| sdague | jaypipes: ok, you need to swing around with your +2 again, as dansmith had to fix the fix | 21:27 |
| jaypipes | sdague: done | 21:29 |
| sdague | \o/ ... ok, time to go make dinner. See you all tomorrow | 21:29 |
| *** ametts has quit IRC | 21:30 | |
| mriedem | alaski: rax doesn't support floating IPs right? | 21:32 |
| *** rfolco has quit IRC | 21:32 | |
| *** rtheis has quit IRC | 21:33 | |
| jroll | mriedem: nope, not today | 21:34 |
| mriedem | ok, was just reading this 'Launch instance with Floating IP' dev list thread | 21:34 |
| *** rmart04 has left #openstack-nova | 21:34 | |
| mriedem | about industry standards | 21:34 |
| clarkb | also no security groups though thats orthogonal to the thread youare talking about | 21:35 |
| *** thorst has joined #openstack-nova | 21:37 | |
| *** annegentle has quit IRC | 21:39 | |
| mriedem | industry standard | 21:41 |
| mriedem | 'nuff said | 21:41 |
| *** moshele has quit IRC | 21:42 | |
| *** pradk has quit IRC | 21:43 | |
| * dansmith is "industry standard" | 21:45 | |
| *** thorst has quit IRC | 21:45 | |
| *** diogogmt has quit IRC | 21:45 | |
| *** liang_ has quit IRC | 21:45 | |
| *** esmiurium has quit IRC | 21:46 | |
| jaypipes | if I'm ever referred to as "industry standard", it is dansmith's sworn oath to shoot me in the head with a grenade launcher. | 21:46 |
| dansmith | wow, that may be a little overkill | 21:47 |
| dansmith | I think a small-calibre handgun would be plenty | 21:47 |
| jaypipes | dansmith: it's an industry standard response, really. | 21:47 |
| *** esmiurium has joined #openstack-nova | 21:47 | |
| * dansmith checks on the industry standard for executions | 21:47 | |
| *** tri2sing has quit IRC | 21:49 | |
| *** tri2sing has joined #openstack-nova | 21:50 | |
| *** Sukhdev has quit IRC | 21:51 | |
| *** stevemar has quit IRC | 21:52 | |
| *** Sukhdev has joined #openstack-nova | 21:55 | |
| *** Sukhdev has quit IRC | 21:59 | |
| *** burt has quit IRC | 22:00 | |
| *** zz_dimtruck is now known as dimtruck | 22:02 | |
| *** annegentle has joined #openstack-nova | 22:02 | |
| *** gyee has joined #openstack-nova | 22:05 | |
| *** BjoernT has quit IRC | 22:05 | |
| *** kylek3h has joined #openstack-nova | 22:06 | |
| *** Sukhdev has joined #openstack-nova | 22:09 | |
| *** tri2sing has quit IRC | 22:10 | |
| *** tri2sing has joined #openstack-nova | 22:11 | |
| *** tri2sing has quit IRC | 22:11 | |
| *** baoli has quit IRC | 22:12 | |
| *** baoli has joined #openstack-nova | 22:13 | |
| *** kfarr has quit IRC | 22:13 | |
| *** su_zhang has quit IRC | 22:14 | |
| *** liangy has joined #openstack-nova | 22:15 | |
| jroll | grenade will just bounce off your head and blow off your legs, though :( | 22:15 |
| jroll | :P | 22:16 |
| *** priteau has quit IRC | 22:16 | |
| *** xyang1 has quit IRC | 22:17 | |
| jaypipes | jroll: honestly, after my day today, even that would be a welcome event. | 22:17 |
| jroll | :( | 22:18 |
| *** eharney has quit IRC | 22:18 | |
| *** mdrabe has quit IRC | 22:22 | |
| *** priteau has joined #openstack-nova | 22:22 | |
| *** stevemar has joined #openstack-nova | 22:24 | |
| *** stevemar has quit IRC | 22:26 | |
| *** marst has quit IRC | 22:26 | |
| *** _ducttape_ has joined #openstack-nova | 22:26 | |
| *** priteau has quit IRC | 22:26 | |
| *** stevemar has joined #openstack-nova | 22:26 | |
| openstackgerrit | Eric Brown proposed openstack/nova: VMware: deprecate wsdl_location conf option https://review.openstack.org/342470 | 22:26 |
| *** liangy has quit IRC | 22:27 | |
| *** tblakes has quit IRC | 22:28 | |
| *** ducttape_ has quit IRC | 22:29 | |
| *** stevemar has quit IRC | 22:31 | |
| *** liangy has joined #openstack-nova | 22:32 | |
| *** krtaylor has joined #openstack-nova | 22:32 | |
| *** dimtruck is now known as zz_dimtruck | 22:34 | |
| *** yamahata has quit IRC | 22:40 | |
| *** thorst has joined #openstack-nova | 22:43 | |
| *** annegentle has quit IRC | 22:46 | |
| *** zz_dimtruck is now known as dimtruck | 22:47 | |
| *** thorst has quit IRC | 22:49 | |
| *** yamahata has joined #openstack-nova | 22:49 | |
| *** thorst has joined #openstack-nova | 22:49 | |
| *** akshai has quit IRC | 22:53 | |
| *** baoli has quit IRC | 22:58 | |
| *** krtaylor has quit IRC | 23:06 | |
| *** stevemar has joined #openstack-nova | 23:07 | |
| *** mingyu has joined #openstack-nova | 23:08 | |
| *** mingyu has quit IRC | 23:10 | |
| *** ayoung has joined #openstack-nova | 23:10 | |
| *** sdake has joined #openstack-nova | 23:10 | |
| *** marst has joined #openstack-nova | 23:12 | |
| *** ljxiash has joined #openstack-nova | 23:13 | |
| harlowja | qq guys, i'm trying to find a good set of links to share with folks on the topic of (what is cells v1); what do u guys think the best source is? (even though i know its cells v1...) | 23:18 |
| *** ljxiash has quit IRC | 23:18 | |
| *** markvoelker has quit IRC | 23:19 | |
| harlowja | said folks are going to try to debug some of the sync issues, so any links people have might be useful (vs me just searching around, ha) | 23:20 |
| *** jamesdenton has joined #openstack-nova | 23:21 | |
| *** akshai has joined #openstack-nova | 23:22 | |
| *** jaypipes has quit IRC | 23:25 | |
| *** jamesdenton has quit IRC | 23:26 | |
| *** mriedem has quit IRC | 23:30 | |
| *** mriedem has joined #openstack-nova | 23:32 | |
| *** dimtruck is now known as zz_dimtruck | 23:32 | |
| *** akshai has quit IRC | 23:33 | |
| *** bapalm has quit IRC | 23:35 | |
| openstackgerrit | melanie witt proposed openstack/nova: Make req spec and RT aware of volume-backed instances https://review.openstack.org/355091 | 23:38 |
| *** Sukhdev has quit IRC | 23:40 | |
| *** rama_y has quit IRC | 23:40 | |
| *** bapalm has joined #openstack-nova | 23:41 | |
| *** raunak has quit IRC | 23:41 | |
| *** zz_dimtruck is now known as dimtruck | 23:46 | |
| *** mingyu has joined #openstack-nova | 23:48 | |
| *** Jeffrey4l has joined #openstack-nova | 23:49 | |
| *** Jeffrey4l has quit IRC | 23:55 | |
| *** markvoelker has joined #openstack-nova | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!