Thursday, 2016-10-06

openstackgerritKen'ichi Ohmichi proposed openstack/nova: Add error handling for delete-volume API  https://review.openstack.org/38266000:00
*** rama_y has quit IRC00:00
*** jamesden_ has quit IRC00:07
*** dikonoor has joined #openstack-nova00:09
*** ijw has joined #openstack-nova00:11
*** ducttape_ has quit IRC00:12
*** kaisers has joined #openstack-nova00:13
*** baoli has quit IRC00:15
*** ijw has quit IRC00:16
*** kaisers has quit IRC00:17
*** dikonoor has quit IRC00:18
*** pradk has joined #openstack-nova00:19
*** markvoelker has joined #openstack-nova00:22
*** stewie925 has quit IRC00:24
*** markvoelker has quit IRC00:27
*** ducttape_ has joined #openstack-nova00:37
*** thorst_ has joined #openstack-nova00:44
*** hoangcx has joined #openstack-nova00:45
oomichialex_xu: mikal: maybe now is your tz. Can you take a look at https://review.openstack.org/#/c/351962/ ?00:46
oomichiI am not sure why this happens..00:46
*** rfolco has quit IRC00:48
*** Pavo has quit IRC00:51
*** anush has quit IRC00:52
*** baoli has joined #openstack-nova00:54
*** dave-mcc_ has joined #openstack-nova00:54
*** zul has joined #openstack-nova00:55
*** dave-mccowan has quit IRC00:55
*** Pavo has joined #openstack-nova00:56
*** Hosam has quit IRC00:57
*** Hosam has joined #openstack-nova00:57
*** Apoorva_ has joined #openstack-nova00:58
*** phuongnh has joined #openstack-nova01:00
*** Apoorva has quit IRC01:01
*** Hosam has quit IRC01:01
*** Apoorva_ has quit IRC01:03
*** raunak has quit IRC01:04
*** browne has quit IRC01:04
*** zul has quit IRC01:08
*** kberger has joined #openstack-nova01:09
*** zul has joined #openstack-nova01:12
*** ijw has joined #openstack-nova01:13
*** code-R has joined #openstack-nova01:14
*** Sukhdev has quit IRC01:15
*** ijw has quit IRC01:18
*** rfolco has joined #openstack-nova01:21
*** TravT has quit IRC01:22
*** unicell has quit IRC01:25
*** dave-mccowan has joined #openstack-nova01:28
*** dave-mcc_ has quit IRC01:29
*** suresh12 has quit IRC01:32
*** suresh12 has joined #openstack-nova01:33
*** suresh12 has quit IRC01:33
*** code-R_ has joined #openstack-nova01:33
*** code-R has quit IRC01:36
*** hemanthm is now known as hemanthm|away01:54
*** tuanluong has joined #openstack-nova01:55
openstackgerritsean mooney proposed openstack/nova: convert libvirt driver to use os-vif for vhost-user with ovs.  https://review.openstack.org/33404802:00
*** BobBall has joined #openstack-nova02:01
*** dave-mccowan has quit IRC02:01
*** BobBall is now known as Guest6909302:01
*** ducnc has quit IRC02:01
*** kaisers has joined #openstack-nova02:02
*** code-R_ has quit IRC02:03
*** code-R has joined #openstack-nova02:03
*** suresh12 has joined #openstack-nova02:04
*** suresh12 has quit IRC02:05
*** baoli has quit IRC02:05
*** kaisers has quit IRC02:06
*** Guest69093 has quit IRC02:06
*** gouthamr has quit IRC02:07
*** code-R_ has joined #openstack-nova02:10
*** amotoki has joined #openstack-nova02:12
*** ducnc has joined #openstack-nova02:13
*** code-R has quit IRC02:14
*** ijw has joined #openstack-nova02:14
*** mriedem has quit IRC02:18
*** ijw has quit IRC02:20
*** BobBall_ has joined #openstack-nova02:20
*** BobBall_ has quit IRC02:24
*** ducnc has quit IRC02:24
*** rfolco has quit IRC02:25
*** thorst_ has quit IRC02:25
*** BobBall_ has joined #openstack-nova02:26
*** thorst_ has joined #openstack-nova02:26
*** winston-d_ has joined #openstack-nova02:29
*** BobBall_ has quit IRC02:31
openstackgerritsean mooney proposed openstack/os-vif: add support for vhost-user reconnect  https://review.openstack.org/35755502:31
*** BobBall_ has joined #openstack-nova02:32
*** ducnc has joined #openstack-nova02:33
*** thorst_ has quit IRC02:35
*** code-R has joined #openstack-nova02:37
*** code-R_ has quit IRC02:37
*** kylek3h has quit IRC02:37
*** BobBall_ has quit IRC02:37
*** haplo37_ has quit IRC02:39
*** kylek3h has joined #openstack-nova02:39
*** BobBall_ has joined #openstack-nova02:39
*** haplo37_ has joined #openstack-nova02:41
*** tongli has quit IRC02:43
*** tiendc has joined #openstack-nova02:43
*** thorst_ has joined #openstack-nova02:47
*** thorst_ has quit IRC02:47
*** thorst_ has joined #openstack-nova02:48
*** Pavo has quit IRC02:51
*** BobBall_ has quit IRC02:51
*** ducttape_ has quit IRC02:51
*** BobBall has joined #openstack-nova02:52
*** BobBall is now known as Guest5454002:52
*** thorst_ has quit IRC02:53
*** hieulq has joined #openstack-nova02:53
*** maticue has quit IRC02:55
*** Pavo has joined #openstack-nova02:55
*** diana-test has quit IRC02:56
*** diana-test has joined #openstack-nova02:57
*** Guest54540 has quit IRC03:02
*** phschwartz has quit IRC03:03
*** BobBall_ has joined #openstack-nova03:03
*** mordred has quit IRC03:03
*** hieulq has quit IRC03:04
*** mordred has joined #openstack-nova03:06
*** phschwartz has joined #openstack-nova03:07
*** BobBall_ has quit IRC03:12
*** yamahata has quit IRC03:12
*** sdake has joined #openstack-nova03:14
*** BobBall_ has joined #openstack-nova03:14
*** ijw has joined #openstack-nova03:16
*** hieulq has joined #openstack-nova03:18
*** ijw has quit IRC03:21
*** hieulq has quit IRC03:38
*** BobBall_ has quit IRC03:38
*** armax has quit IRC03:41
*** annegentle has quit IRC03:42
*** code-R has quit IRC03:43
*** sdake_ has joined #openstack-nova03:45
*** sdake has quit IRC03:47
*** BobBall has joined #openstack-nova03:47
*** armax has joined #openstack-nova03:47
*** links has joined #openstack-nova03:47
*** BobBall is now known as Guest64203:48
*** kaisers has joined #openstack-nova03:50
*** thorst_ has joined #openstack-nova03:51
*** code-R has joined #openstack-nova03:52
*** Guest642 has quit IRC03:53
*** code-R_ has joined #openstack-nova03:53
*** BobBall_ has joined #openstack-nova03:55
*** kaisers has quit IRC03:55
*** rama_y has joined #openstack-nova03:55
*** code-R has quit IRC03:57
*** thorst_ has quit IRC03:58
*** BobBall_ has quit IRC04:00
*** armax has quit IRC04:02
*** armax has joined #openstack-nova04:02
*** armax has quit IRC04:03
*** mtanino has quit IRC04:03
*** John341 has quit IRC04:06
*** haplo37 has quit IRC04:07
*** GB21 has joined #openstack-nova04:11
*** agireud has quit IRC04:16
*** haplo37 has joined #openstack-nova04:17
*** ijw has joined #openstack-nova04:17
*** ijw has quit IRC04:23
*** agireud has joined #openstack-nova04:24
*** Sukhdev has joined #openstack-nova04:25
*** markvoelker has joined #openstack-nova04:25
*** jaosorior has joined #openstack-nova04:25
*** GB21 has quit IRC04:26
*** markvoelker has quit IRC04:30
*** unicell has joined #openstack-nova04:32
*** unicell has quit IRC04:33
*** unicell has joined #openstack-nova04:36
*** sudswas__ has joined #openstack-nova04:36
*** sudipto has joined #openstack-nova04:36
*** code-R has joined #openstack-nova04:37
*** code-R_ has quit IRC04:37
*** GB21 has joined #openstack-nova04:38
*** code-R_ has joined #openstack-nova04:41
*** salv-orlando has joined #openstack-nova04:41
*** sudipto has quit IRC04:42
*** sudswas__ has quit IRC04:42
*** code-R has quit IRC04:44
*** liverpooler has quit IRC04:45
*** salv-orlando has quit IRC04:46
*** zz_dimtruck is now known as dimtruck04:47
*** ratailor has joined #openstack-nova04:51
*** Pavo has quit IRC04:51
*** psachin has joined #openstack-nova04:51
*** unicell has quit IRC04:51
*** unicell1 has joined #openstack-nova04:51
*** yamahata has joined #openstack-nova04:52
*** thorst_ has joined #openstack-nova04:55
*** Pavo has joined #openstack-nova04:55
*** winston-d_ has quit IRC04:56
*** adisky has joined #openstack-nova04:56
*** narayrak has joined #openstack-nova04:56
*** narayrak has quit IRC04:59
*** sridharg has joined #openstack-nova04:59
*** thorst_ has quit IRC05:02
*** code-R has joined #openstack-nova05:07
*** code-R_ has quit IRC05:07
*** salv-orlando has joined #openstack-nova05:07
*** takashin has joined #openstack-nova05:10
*** pbandark has joined #openstack-nova05:12
*** code-R has quit IRC05:12
*** haplo37_ has quit IRC05:15
*** takashin has quit IRC05:16
*** takashin has joined #openstack-nova05:17
*** haplo37_ has joined #openstack-nova05:17
*** TravT has joined #openstack-nova05:18
*** ijw has joined #openstack-nova05:19
*** david-lyle has joined #openstack-nova05:19
*** prateek has joined #openstack-nova05:21
*** ijw has quit IRC05:24
*** markvoelker has joined #openstack-nova05:26
*** markvoelker has quit IRC05:31
*** ldeptula has joined #openstack-nova05:38
*** John341 has joined #openstack-nova05:39
*** kaisers has joined #openstack-nova05:39
*** oanson has joined #openstack-nova05:41
*** dimtruck is now known as zz_dimtruck05:43
*** alex_xu has quit IRC05:48
*** kaisers has quit IRC05:48
*** sahid has joined #openstack-nova05:50
*** sahid has quit IRC05:51
*** alex_xu has joined #openstack-nova05:51
*** sahid has joined #openstack-nova05:51
openstackgerritsean mooney proposed openstack/os-vif: remove use of contextlib and with nested  https://review.openstack.org/38272505:51
*** Sukhdev has quit IRC05:51
*** kaisers has joined #openstack-nova05:55
*** jaosorior has quit IRC05:55
*** suresh12 has joined #openstack-nova05:55
*** jaosorior has joined #openstack-nova05:57
*** diga has joined #openstack-nova05:58
*** thorst_ has joined #openstack-nova06:00
*** liverpooler has joined #openstack-nova06:02
*** david-lyle_ has joined #openstack-nova06:04
*** david-lyle has quit IRC06:04
*** nkrinner_afk is now known as nkrinner06:04
*** thorst_ has quit IRC06:07
*** liverpooler has quit IRC06:07
*** liverpooler has joined #openstack-nova06:07
*** paul-carlton2 has joined #openstack-nova06:10
*** sdake_ has quit IRC06:11
*** david-lyle_ has quit IRC06:12
*** code-R has joined #openstack-nova06:14
*** moshele has joined #openstack-nova06:15
*** rcernin has joined #openstack-nova06:15
*** code-R_ has joined #openstack-nova06:16
*** hussain_c has quit IRC06:17
*** code-R has quit IRC06:19
*** ijw has joined #openstack-nova06:20
*** lpetrut has joined #openstack-nova06:22
sahidjaypipes: hello, any chance you ACK that one https://review.openstack.org/#/c/378560/06:22
sahidit's real issue related to live migrate instances with ephemeral disks06:22
openstackgerritDivya K Konoor proposed openstack/nova-specs: Allow admin users to pass key_name corresponding to another user and deploy a VM successfully.  https://review.openstack.org/38258806:25
*** ijw has quit IRC06:26
*** markvoelker has joined #openstack-nova06:27
*** salv-orlando has quit IRC06:29
*** markvoelker has quit IRC06:32
*** hshiina has joined #openstack-nova06:32
*** andreas_s has joined #openstack-nova06:33
*** salv-orlando has joined #openstack-nova06:36
*** salv-orlando has quit IRC06:41
*** zenoway has joined #openstack-nova06:41
*** rama_y has quit IRC06:43
*** haplo37 has quit IRC06:51
*** Pavo has quit IRC06:51
*** lpetrut has quit IRC06:51
*** haplo37 has joined #openstack-nova06:51
*** jichen has joined #openstack-nova06:51
*** Pavo has joined #openstack-nova06:56
*** suresh12 has quit IRC06:57
*** unicell has joined #openstack-nova06:57
*** unicell1 has quit IRC06:58
*** ekuris has joined #openstack-nova07:03
*** code-R_ has quit IRC07:03
*** thorst_ has joined #openstack-nova07:05
*** hoangcx has quit IRC07:06
*** lpetrut has joined #openstack-nova07:07
*** ihrachys has joined #openstack-nova07:08
*** ecelik has joined #openstack-nova07:08
*** markus_z has joined #openstack-nova07:09
*** tesseract- has joined #openstack-nova07:11
*** psachin has quit IRC07:12
*** thorst_ has quit IRC07:12
*** ecelik has left #openstack-nova07:12
*** Dinesh_Bhor has joined #openstack-nova07:13
*** code-R has joined #openstack-nova07:13
*** salv-orlando has joined #openstack-nova07:14
*** psachin has joined #openstack-nova07:17
*** ihrachys has quit IRC07:18
*** lpetrut has quit IRC07:21
*** ijw has joined #openstack-nova07:22
*** jlanoux has joined #openstack-nova07:24
*** jlanoux has quit IRC07:25
*** jlanoux has joined #openstack-nova07:26
*** pcaruana has joined #openstack-nova07:26
*** code-R has quit IRC07:27
*** ijw has quit IRC07:28
*** markvoelker has joined #openstack-nova07:28
*** code-R has joined #openstack-nova07:28
openstackgerritDivya K Konoor proposed openstack/nova-specs: Allow admin users to pass key_name corresponding to another user and deploy a VM successfully.  https://review.openstack.org/38258807:30
*** ralonsoh has joined #openstack-nova07:32
*** markvoelker has quit IRC07:32
*** code-R has quit IRC07:34
*** danpawlik has joined #openstack-nova07:43
danpawlikHello folks, can you check PS https://review.openstack.org/#/c/314928/ ?07:43
*** danpb has joined #openstack-nova07:44
*** alexchadin has joined #openstack-nova07:46
*** mvk has quit IRC07:51
*** unicell has quit IRC07:54
*** ihrachys has joined #openstack-nova07:56
*** salv-orl_ has joined #openstack-nova07:56
*** ducttape_ has joined #openstack-nova07:56
*** unicell has joined #openstack-nova07:56
*** salv-orlando has quit IRC07:58
*** zzzeek has quit IRC08:00
*** paul-carlton2 has quit IRC08:00
*** ducttape_ has quit IRC08:01
*** zzzeek has joined #openstack-nova08:01
*** haplo37_ has quit IRC08:01
*** hoangcx has joined #openstack-nova08:03
*** haplo37_ has joined #openstack-nova08:03
*** thorst_ has joined #openstack-nova08:10
*** ihrachys has quit IRC08:11
*** code-R has joined #openstack-nova08:12
*** code-R_ has joined #openstack-nova08:13
*** jichen has quit IRC08:16
*** yassine has joined #openstack-nova08:16
*** code-R has quit IRC08:17
*** yassine is now known as Guest3910908:17
*** thorst_ has quit IRC08:17
*** jaosorior is now known as jaosorior_lunch08:17
*** code-R_ has quit IRC08:18
*** mgoddard has joined #openstack-nova08:19
*** mvk has joined #openstack-nova08:23
*** ijw has joined #openstack-nova08:24
*** ijw has quit IRC08:29
*** lpetrut has joined #openstack-nova08:31
*** Matti_ has quit IRC08:35
*** derekh has joined #openstack-nova08:38
*** ihrachys has joined #openstack-nova08:39
*** suresh12 has joined #openstack-nova08:43
*** suresh12 has quit IRC08:48
*** sridharg has quit IRC08:50
*** Pavo has quit IRC08:51
*** Pavo has joined #openstack-nova08:56
*** ociuhandu has joined #openstack-nova08:56
*** ducttape_ has joined #openstack-nova08:57
*** alexchad_ has joined #openstack-nova08:57
*** alexchadin has quit IRC09:00
*** takashin has left #openstack-nova09:01
*** ducttape_ has quit IRC09:02
mdboothkashyap: Did you get anywhere interesting with the rebase thing last night?09:02
kashyapmdbooth: After 2 hours of last night, ended up with this - https://bugzilla.redhat.com/show_bug.cgi?id=138216509:02
openstackbugzilla.redhat.com bug 1382165 in libvirt "virDomainGetBlockJobInfo: Adjust job reporting based on QEMU stats & the "ready" field of `query-block-jobs`" [Unspecified,New] - Assigned to libvirt-maint09:02
kashyapmdbooth: I wrote an email to - http://lists.openstack.org/pipermail/openstack-dev/2016-October/105158.html09:02
mdboothkashyap: Reading, thanks. Ah... just saw your mail.09:02
*** sridharg has joined #openstack-nova09:04
*** ralonsoh_ has joined #openstack-nova09:04
kashyapmdbooth: Also read the small IRC snippet w/ Eric, on the above bug.09:05
mdboothkashyap: That's an extremely thorough bz09:05
*** ralonsoh has quit IRC09:05
kashyapmdbooth: Yeah, spent way too much time than I care to admit last night :P09:05
kashyapThx.09:06
csatariHi, can someone help me in api-ref tox-ing? It has problems with the sidebar_mode.09:06
kashyapIt was nice how Eric thought out loud about "first I need to be convinced that this is a real race"09:06
kashyapThen we reached the conclusion :-)09:06
mdboothkashyap: I'm not a huge fan of libvirt fudging the progress numbers, I have to say.09:07
kashyapmdbooth: Hmm, I thought about it too, briefly.  But wondered what else options are there09:08
mdboothYeah, I guess the issue is not breaking existing users who made assumptions which were valid at the time.09:09
kashyapmdbooth: Exactly.  If we know that QEMU is new enough, then, we could wait for that event rather than polling09:09
mdboothkashyap: So, if we switch to listening for events, would it fix this?09:09
mdboothThe event is generated by qemu, not libvirt, right?09:09
mdboothIncidentally, what do other hypervisors do here?09:10
kashyapmdbooth: Yes, the event BLOCK_JOB_READY is a QMP event09:10
mdboothWe can see that event in the libvirt logs from the failed test run, btw09:11
mdboothBut 200ms *after* we tried to pivot09:11
kashyapmdbooth: Exactly.  That's what I pointed out in step (6)09:11
kashyapIn the email09:11
* mdbooth has only read the bz so far09:11
mdboothkashyap: Sorry, let me catch up with you :)09:11
kashyapmdbooth: Yeah, please read the email analysis too.  I wrote that just for you09:12
*** karthiks has joined #openstack-nova09:13
*** thorst_ has joined #openstack-nova09:14
*** sridharg has quit IRC09:15
*** sridharg has joined #openstack-nova09:16
*** Matti_ has joined #openstack-nova09:18
*** ihrachys has quit IRC09:19
mdboothkashyap: That's excellent, thanks.09:19
mdboothkashyap: So, are there any downsides if we switch to handling completion events?09:20
mdboothWould it work with non-qemu hypervisors?09:20
mdboothdanpb: ^^^09:20
kashyapmdbooth: I don't know of any downsides at the moment.09:20
kashyapNot sure about the non-QEMU hypervisors, good question.09:20
danpbmdbooth: completion events for what ?09:21
mdboothkashyap: polling for completion just looks like a quagmire of workarounds09:21
mdboothdanpb: in this case, block jobs09:21
danpbmdbooth: non-qemu is irrelevant then - qemu is the only thing doing block jobs09:21
mdboothkashyap: So, sounds like we should switch to handling completion events09:22
*** thorst_ has quit IRC09:22
mdboothdanpb: How about support from Ubuntu's paleolithic libvirt/qemu versions?09:22
*** zenoway has quit IRC09:22
kashyapmdbooth: Yeah.  That seems to be the clearest indication09:22
kashyapHehe, "paleolithic" :D09:23
danpbmdbooth: no idea09:23
*** mnestratov has joined #openstack-nova09:23
mdboothdanpb: Are we likely to hit issues with old versions which didn't generate certain events?09:23
danpbno idea09:23
mdboothdanpb: Ok, thanks.09:24
kashyapmdbooth: Here's what you're looking for -- the QAPI schema doc for that event says, it is available since QEMU 1.309:24
kashyapSee line 2670 --  http://git.qemu.org/?p=qemu.git;a=blob;f=qapi/block-core.json;h=9d797b8fe0050443eb6393cbda77e126ac974ed3;hb=HEAD#l265109:24
mdboothkashyap: I think we only need COMPLETED, FAILED, CANCELLED, and READY, right?09:24
mdboothkashyap: Your git-fu is very strong, btw.09:25
*** ijw has joined #openstack-nova09:25
*** code-R has joined #openstack-nova09:25
kashyapmdbooth: Don't let it give such an impression, I can't do Octopus merges09:25
* mdbooth neither, but I do know what one is ;)09:26
kashyapmdbooth: About what we need.  Incidentally John Snow, from QEMU, was just debating yesterday09:26
kashyapTrying to answer the question: "Whether jobs should always emit events?"09:26
openstackgerritsahid proposed openstack/nova: libvirt: add event lifecycle details  https://review.openstack.org/37394309:27
*** zenoway has joined #openstack-nova09:27
mdboothkashyap: There needs to be an api we can rely on.09:27
kashyapYes09:28
mdboothRight now, I'm convinced polling job info isn't it.09:28
mdboothWell, almost convinced.09:28
* kashyap is on a coffee shop n/w, and every half-an-hour this has some stupid captive-portal nonsense. Wonder how I could subvert it09:28
mdboothI'm slightly nervous about events, because a missing event would be disasterous.09:28
*** mvk has quit IRC09:29
*** code-R_ has joined #openstack-nova09:29
kashyapmdbooth: Indeed, maybe we could have a "two-phased approach"09:29
mdboothThat also lacks simplicity.09:30
mdboothLikely we'd get the union of both sets of bugs :)09:30
kashyapLOL, union of both sets09:30
*** ociuhandu has quit IRC09:31
*** ijw has quit IRC09:31
kashyapmdbooth: Currently, I see these possible QMP events: BLOCK_IO_ERROR, BLOCK_JOB_CANCELLED, BLOCK_JOB_COMPLETED, BLOCK_JOB_ERROR, BLOCK_JOB_READY09:31
mdboothkashyap: I'm just trying to decide if we need to handle IO_ERROR09:31
mdbooth # Note: If action is "stop", a STOP event will eventually follow the09:32
mdbooth2564 # BLOCK_IO_ERROR event09:32
*** code-R has quit IRC09:32
*** hshiina has quit IRC09:33
kashyapYeah, that's certainly useful - what action to take when read / write IO errors occur09:33
mdboothkashyap: So, it looks like fatal IO_ERROR will be followed by BLOCK_JOB_ERROR09:34
mdboothIoW, we wouldn't want to handle IO_ERROR directly09:35
kashyapmdbooth: Yes09:35
mdboothExcept maybe to log it09:35
*** jaosorior_lunch has quit IRC09:36
*** garyk has joined #openstack-nova09:36
*** jaosorior_lunch has joined #openstack-nova09:36
kashyapDefinitely.  mdbooth -- I'm also wondering if / how it interacts with the QEMU command-line 'werror/rerror'09:37
kashyap           werror=action,rerror=action09:37
kashyap               Specify which action to take on write and read errors. Valid actions are: "ignore" (ignore the error and try to continue), "stop" (pause QEMU), "report" (report the error09:37
kashyap               to the guest), "enospc" (pause QEMU only if the host disk is full; report the error to the guest otherwise).  The default setting is werror=enospc and rerror=report.09:37
*** gszasz has joined #openstack-nova09:37
* kashyap just pasted that from QEMU man page09:37
mdboothkashyap: So, fixing this is going to require a significant reworking of this code, tricky dev setups, completely new unit tests, etc.09:38
garykdanpb: mdbooth: maybe you guys can help out here. i am running stable/newton with 3 kvm nodes and have hit the following: https://bugs.launchpad.net/nova/+bug/1630912 and https://bugs.launchpad.net/nova/+bug/163091809:38
openstackLaunchpad bug 1630912 in OpenStack Compute (nova) "stable/newton: unable to spin up a kvm instance" [Critical,New]09:38
openstackLaunchpad bug 1630918 in OpenStack Compute (nova) "stable/new: resource tracker exception" [Critical,New]09:38
kashyapmdbooth: Yes, probably a spec proposal required for this?09:38
mdboothkashyap: I was just looking for a way to get the immediate issue off our plate.09:38
kashyapmdbooth: Is that gate-blocking?09:38
mdboothkashyap: Nah, this is a bug without a wider impact. Spec is total overkill imho.09:39
mdboothAnyway, the way to do that would be to handle the Ready flag, I think09:39
mdboothBUT....09:39
mdboothlibvirt doesn't make it available to us09:39
kashyapSure. I'm not eager for it.  Just checking the criteria09:39
mdboothIt's not just the python bindings, either09:39
mdboothLook at struct _virDomainBlockJobInfo09:40
mdboothIt's just type, bandwidth, cur, end.09:40
*** mvk has joined #openstack-nova09:41
kashyapmdbooth: Yes, it's not.  I just asked on #virt, OFTC to a libvirt dev if they think exposing the 'ready' flag is a reasonable idea09:41
mdboothSo at first glance, I don't think it's possible to fix this at all with the current interface.09:41
mdboothExcept by moving to handling events.09:41
kashyapmdbooth: You know libvirt internally seem to use VIR_DOMAIN_BLOCK_JOB_READY (which is just a translation of QEMU's BLOCK_JOB_READY) I guess09:43
mdboothAnd given that we need to 'support' the unsupported Ubuntu virt stack, which will never receive bug fixes, I think we need to JFDI.09:43
kashyapmdbooth: There we go:09:44
kashyap11:41 < kashyap> mprivozn: When you get a moment, is it reasonble to request libvirt to expose the the "ready" flag (from QMP `query-block-jobs`) via its bindings?09:44
kashyap11:42  * kashyap should probably bring this up on the mailing list09:44
kashyap11:42 < mprivozn> kashyap: yeah; but my answer here is that it's not possible with the current API because of ABI stability :(09:44
kashyap11:43 < mprivozn> kashyap: but please do initiate discussion on the list09:44
*** jaosorior_lunch is now known as jaosorior09:44
mdboothkashyap: ABI stability is moot when you've got total stability because the stack is unmaintained.09:44
mdboothI don't see any option other than to switch to the other API09:45
mdboothgaryk: Looking09:46
kashyapmdbooth: Yeah.  mdbooth Also: The immediate Nova issue at hand, is it a Gate-blocking one?  (I suppose so, if it's enabled in Tempest?)09:46
mdboothkashyap: Nah, it's a new test, right?09:46
mdboothSo this has always been broken, we're just discovering it now.09:47
kashyapmdbooth: Oh, right.  It's a new test, reading Matt R's commit message09:47
mdboothgaryk: Fun!09:48
mdboothgaryk: Are they related, btw?09:48
*** aarefiev has quit IRC09:50
kashyap"04709c94-0ba3-4312-ad1c-5739e5600bf4] IOError: [Errno 13] Permission denied: '/opt/stack/data/nova/instances/locks/nova-273d5645757056cdd056de4cfe9f121b9eee6ae3'"09:50
*** thorst_ has joined #openstack-nova09:50
*** ralonsoh_ is now known as ralonsoh09:50
*** roeyc has joined #openstack-nova09:50
*** aarefiev has joined #openstack-nova09:51
*** salv-orl_ has quit IRC09:51
mdboothkashyap: Looks like it's in spawn trying to lock presumably the image in the image cache09:51
openstackgerritjichenjc proposed openstack/nova-specs: Add host-command-with-POST  https://review.openstack.org/30338809:52
*** kaisers_ has joined #openstack-nova09:54
*** kaisers_ has quit IRC09:55
*** ducttape_ has joined #openstack-nova09:57
*** thorst_ has quit IRC09:58
*** kaisers has quit IRC09:58
*** kaisers has joined #openstack-nova09:58
*** cdent has joined #openstack-nova09:59
*** Margarita_S has joined #openstack-nova09:59
*** woodster_ has quit IRC10:00
*** kaisers has quit IRC10:00
*** ihrachys has joined #openstack-nova10:02
openstackgerritjichenjc proposed openstack/nova-specs: Add host-command-with-POST  https://review.openstack.org/30338810:02
*** ducttape_ has quit IRC10:02
*** ociuhandu has joined #openstack-nova10:04
cdentrpodolyaka: in response to your most recent comment on the placement aggregates: https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/migrate_repo/versions/318_resource_provider_name_aggregates.py#L3310:06
mdboothgaryk: Responded to the first with a brain dump10:06
cdentso yeah, messy10:06
rpodolyakacdent: ah, I see. Thanks! Yeah, kind of messy. But I guess it's fine as long as we do not need to change the charset of existing data in tables10:09
cdentthe expectation is that that table will never change since it just mapping id to uuid but ... the future is foggy10:10
*** kaisers has joined #openstack-nova10:10
*** Matti_ has quit IRC10:10
*** hoangcx has quit IRC10:11
garykmdbooth: thanks! ithink that they are unrelated but in the same env… it will take a while for stable to actaully become stable10:12
*** hoangcx has joined #openstack-nova10:12
openstackgerritjichenjc proposed openstack/nova-specs: Add host-command-with-POST  https://review.openstack.org/30338810:12
mdboothgaryk: Just taking a quick look at the second.10:12
mdboothLooks like bdms doesn't contain the instance uuid10:12
*** hoangcx has quit IRC10:12
mdboothSo... we've got an instance with no bdms? Or a race.10:12
*** kaisers has quit IRC10:14
mdboothSo...10:14
danpawlikHello, pls check https://bugs.launchpad.net/nova/+bug/1630930  and if you have some free time just take a look on other PS https://review.openstack.org/#/c/314928/10:14
openstackLaunchpad bug 1630930 in OpenStack Compute (nova) "Affinity instance slot reservation" [Undecided,New]10:14
mdbooth        bdms = objects.BlockDeviceMappingList.bdms_by_instance_uuid(10:14
mdbooth            ctx, instance_uuids)10:14
mdboothbdms[guest.uuid]10:14
mdboothCan we do indexed access to BlockDeviceMappingList?10:15
mdboothAh, nm: base.obj_make_dict_of_lists10:17
kashyapWow, this has to be the "Month of Race Conditions" or something10:19
garykmdbooth: i am not attaching any volumes10:19
mdboothkashyap: Race conditions are always in season.10:20
mdboothgaryk: I think there should be a BDM for at least the root disk10:21
mdboothgaryk: Also, this code has been around a while, so perhaps something changed elsewhere...10:22
mdboothgaryk: Do you have a slave configured?10:22
*** salv-orlando has joined #openstack-nova10:23
*** code-R_ has quit IRC10:24
garykmdbooth: not aware of any slave configured10:24
mdboothThat looks irrelevant anyway.10:24
*** code-R has joined #openstack-nova10:24
mdboothAre these running in a CI, btw?10:24
mdboothCan you link to full logs?10:24
*** riccardo_ has joined #openstack-nova10:25
garykmdbooth: no, its not in CI. its an internal test enviroment that we build. 3 KVM nodes and one VC node.10:26
openstackgerritChris Dent proposed openstack/nova: A demo script for updating resource inventory  https://review.openstack.org/38261310:26
mdboothgaryk: Anything weird going on? Concurrent deletes, maybe?10:26
garykmdbooth: prblem started when we saw that some KVM instances had 2 or 3 IP's.10:26
*** ijw has joined #openstack-nova10:26
garykmdbooth: no. also happens when i spin up one VM10:27
riccardo_hi guys, where in the nova code can I find the place where resource allocation is done?10:27
*** kaisers has joined #openstack-nova10:27
garykmdbooth: its pretty random. still trying to iron it down10:27
mdboothgaryk: Not the second one though, right?10:27
garykmdbooth: problem is also when there is a rechedule the instance ports are not cleand up10:27
garykmdbooth: nop - only one vnic10:28
mdboothSecond bug is in get_available_resource()10:28
mdboothThat's isn't called during spawn(), is it?10:28
*** zenoway has quit IRC10:29
*** zenoway has joined #openstack-nova10:31
*** ijw has quit IRC10:32
openstackgerritgaryk proposed openstack/nova: libvirt: ensure that BDM exists  https://review.openstack.org/38283310:32
garykmdbooth: correct - that is related to the resource tracker. those exceptions are showing up randomly10:32
garyki posted something above - still need to figure out how to unit test this10:33
mdboothgaryk: Are you using NFS, btw?10:33
garykmdbooth: not that i am aware of10:35
garykit should all be local10:35
*** tiendc has quit IRC10:35
mdboothJusting wondering about the first thing. If you can get on there in happy hacker mode, might be interesting to get a 'ls -l' from the locks directory immediately before the lock fails.10:36
garykmdbooth: ok, i will try10:36
mdboothgaryk: To get as close as possible, I'd hack the library :)10:37
*** alexchad_ is now known as alexchadin10:37
garykmdbooth: my bad on the NFS - the KVM nodes are usning NFS10:38
garykthat may be the root cause.10:38
mdboothgaryk: Well we should still work on NFS, however it is a huge additional vector for bugs and/or misconfiguration.10:38
openstackgerritStephen Finucane proposed openstack/nova-specs: Add extra-specs-image-meta-conflicts spec  https://review.openstack.org/38238210:38
garykmdbooth: yeah, i hear you.10:39
*** kaisers has quit IRC10:39
*** kaisers has joined #openstack-nova10:40
garykdanpb: mdbooth: any change you guys can please look at https://review.openstack.org/140045. this is nearly 2 years in review10:40
garykjohnthetubaguy: ^10:40
garyki thought that the vmware drivers had a rough time, but this is taking the piss10:40
*** sdague has joined #openstack-nova10:41
garykmdbooth: dumb question - when using NFS, is there a way in which we configure nova to be aware of that. i am really not familiar with this10:42
mdboothgaryk: Nova isn't aware of it. It's just a quirky filesystem.10:43
garykmdbooth: ok, thanks10:43
johnthetubaguyos-vif was blocking that, should be clear to land now that was ironed out last cycle10:43
johnthetubaguythere are plans to make resource tracker "shared storage" aware this cycle10:43
*** alexchadin has quit IRC10:44
mdboothgaryk: Are you trying to spawn multiple instances simultaneously, btw?10:45
garykmdbooth: no, one at a time10:46
garykif i shut down all but one nova compute then all works well. when there are 3 running everything goes pear shaped10:47
garykmay be a setup issue10:47
mdboothgaryk: I was just thinking around concurrent access to the locks directory10:47
mdboothBut again, we support that, so if there was anything architecturally broken around it I'd expect it to have come up before.10:48
garykagree10:48
mdboothgaryk: Does it work with any single nova compute, or 1 specific one?10:48
garykmdbooth: it works with a single nova compute10:48
*** tbachman has quit IRC10:48
mdboothSo I'd guess probably a misconfiguration on the other 2?10:49
sfinucansahid: Any reason not to resuscitate this patch? https://review.openstack.org/#/c/172079/10:49
openstackgerritFeodor Tersin proposed openstack/nova: libvirt: Improve _is_booted_from_volume implementation  https://review.openstack.org/38202410:49
openstackgerritFeodor Tersin proposed openstack/nova: libvirt: Remove unused disk_info parameter  https://review.openstack.org/38202510:49
mdboothOr on the NFS host for the other 2.10:49
garyki just terminated 2 nova computes and it works10:50
garykwhen all 3 are running it does not10:50
johnthetubaguyis that the image cache going crazy? not sure how you configure that on NFS10:50
mdboothkashyap: Incidentally, these docs:10:50
mdboothhttp://libvirt.org/html/libvirt-libvirt-domain.html#virDomainBlockRebase10:50
mdboothkashyap: Say to do exactly what we're doing10:50
mdboothkashyap: So if the outcome is we need to do something else, we also need to fix the docs.10:50
*** Pavo has quit IRC10:51
openstackgerritTakashi NATSUME proposed openstack/nova-specs: Enable cold migration with target host  https://review.openstack.org/33428610:51
*** slaweq has quit IRC10:52
openstackgerritTakashi NATSUME proposed openstack/nova-specs: Enable cold migration with target host  https://review.openstack.org/33428610:53
*** takashin has joined #openstack-nova10:53
*** pjm6 has quit IRC10:54
*** pjm6 has joined #openstack-nova10:54
*** thorst_ has joined #openstack-nova10:54
*** kaisers has quit IRC10:56
*** Pavo has joined #openstack-nova10:56
*** mjura has joined #openstack-nova10:57
openstackgerritEmilien Macchi proposed openstack/nova: Document experimental pipeline in Nova CI  https://review.openstack.org/38183810:58
*** ducttape_ has joined #openstack-nova10:58
*** thorst_ has quit IRC11:02
*** ducttape_ has quit IRC11:03
*** mjura has quit IRC11:04
*** hshiina has joined #openstack-nova11:05
kashyapmdbooth: (Was AFK for lunch)  True, the API docs ought to be fixed, if the outcome is diff.11:05
*** roeyc has quit IRC11:06
*** rtheis has joined #openstack-nova11:08
*** roeyc has joined #openstack-nova11:09
*** paul-carlton2 has joined #openstack-nova11:10
*** Margarita_S has quit IRC11:10
openstackgerritGergely Csatari proposed openstack/nova: Checking the parameters of servers-actions.inc  https://review.openstack.org/32711211:10
*** code-R has quit IRC11:13
openstackgerritPaul Carlton proposed openstack/nova: Update nova network info when doing rebuild for evacuate operation  https://review.openstack.org/38285311:13
*** bvanhav has joined #openstack-nova11:16
*** anthonyper has quit IRC11:17
*** anthonyper has joined #openstack-nova11:17
*** mjura has joined #openstack-nova11:17
*** ralonsoh has quit IRC11:17
*** ralonsoh has joined #openstack-nova11:18
*** ftersin has quit IRC11:20
*** ftersin has joined #openstack-nova11:21
*** code-R has joined #openstack-nova11:26
*** ijw has joined #openstack-nova11:28
openstackgerritJABAR ALI proposed openstack/python-novaclient: Chnages for bug 1613105  https://review.openstack.org/38286411:31
openstackbug 1613105 in python-novaclient "functional tests leak cinder volumes" [Undecided,Confirmed] https://launchpad.net/bugs/1613105 - Assigned to JABAR ALI (jabbar-ali)11:31
*** baoli has joined #openstack-nova11:31
*** hussain_c has joined #openstack-nova11:32
*** code-R has quit IRC11:32
*** nkrinner has quit IRC11:34
*** ijw has quit IRC11:34
*** ducnc has quit IRC11:34
*** phuongnh has quit IRC11:37
sahidsfinucan: it's probably a valid patch, just leak of reviews11:37
sahidlet me try to resusciatete it11:38
*** claudiub has quit IRC11:45
*** thorst_ has joined #openstack-nova11:45
*** nkrinner has joined #openstack-nova11:46
*** alexchadin has joined #openstack-nova11:49
*** gcb has joined #openstack-nova11:49
*** ociuhandu has quit IRC11:52
*** rfolco has joined #openstack-nova11:55
*** ratailor has quit IRC11:56
*** Margarita_S has joined #openstack-nova11:58
*** ducttape_ has joined #openstack-nova11:59
*** thorst__ has joined #openstack-nova12:00
*** prateek has quit IRC12:02
cdentjohnthetubaguy, sdague, jaypipes : a series of bug fixes on placement welcome your attention if you've got the time: https://review.openstack.org/#/c/374800/ https://review.openstack.org/#/c/374833/ https://review.openstack.org/#/c/374870/12:02
*** gouthamr has joined #openstack-nova12:03
*** ducttape_ has quit IRC12:04
*** thorst_ has quit IRC12:04
*** tuanluong has quit IRC12:04
*** thorst__ has quit IRC12:05
*** thorst_ has joined #openstack-nova12:06
*** lucas-afk is now known as lucasagomes12:07
*** thorst__ has joined #openstack-nova12:07
*** mgoddard_ has joined #openstack-nova12:08
*** thorst_ has quit IRC12:11
*** ducttape_ has joined #openstack-nova12:11
*** mgoddard has quit IRC12:11
*** alex_xu has quit IRC12:12
*** thorst__ has quit IRC12:12
*** thorst_ has joined #openstack-nova12:13
*** thorst__ has joined #openstack-nova12:14
cdentthanks sdague12:14
*** alex_xu has joined #openstack-nova12:14
*** mgoddard_ has quit IRC12:15
*** thorst_ has quit IRC12:17
sahidwe have an "easy" fix here related to live migrate instances with ephemeral disks https://review.openstack.org/#/c/378560/12:18
openstackgerritGábor Antal proposed openstack/nova: Removed unnecessary parantheses in return statements  https://review.openstack.org/37054912:19
sahidit has already been +2ed so if we can have the final +2+W it would be nice12:19
*** tblakes has joined #openstack-nova12:19
*** thorst__ has quit IRC12:19
*** Jeffrey4l_ has quit IRC12:20
*** Jeffrey4l_ has joined #openstack-nova12:21
*** prateek has joined #openstack-nova12:21
*** mgoddard has joined #openstack-nova12:22
*** kaisers has joined #openstack-nova12:24
*** salv-orlando has quit IRC12:24
*** lamt has quit IRC12:25
*** maticue has joined #openstack-nova12:25
*** edmondsw has joined #openstack-nova12:26
*** omnipresent has joined #openstack-nova12:27
*** zenoway has quit IRC12:27
*** alex_xu has quit IRC12:27
*** esberglu has joined #openstack-nova12:29
*** alex_xu has joined #openstack-nova12:29
*** ijw has joined #openstack-nova12:29
*** ducttape_ has quit IRC12:30
*** markvoelker has joined #openstack-nova12:31
*** code-R has joined #openstack-nova12:34
garykmdbooth: seems like the file permission on the directory were screwed up.12:34
garykwhen changing them manually to the devstack user then everthing worked12:34
mdboothgaryk: On the locks directory itself?12:34
garykmdbooth: yeah.12:34
mdboothWonder why it worked on one node but not the others12:34
garykjust not sure why they were messed up in the first place12:34
mdboothNFS thing?12:35
mdboothWhat mode did you have to set?12:35
*** ijw has quit IRC12:35
mdboothIf it was 777 you might have a uid mapping issue12:35
garyknot sure. what is concerning is that when there are failures that nova does not free the correct neutron resources. that is the real bug12:35
garyki will try and look into that as that is very concerning12:35
garyklet me check12:36
*** diana-test has joined #openstack-nova12:37
garykmdbooth: it 75512:37
*** diana-test has quit IRC12:40
mdboothgaryk: So interestingly I can't see what creates the locks directory12:40
mdboothI wonder if it has be created manually...12:40
*** peter-hamilton has joined #openstack-nova12:40
kashyapmdbooth: When you about later, turns out that 'ready' state is already exposed via guest XML...See Peter's response on that upstream thread.12:41
openstackgerritSivasathurappan Radhakrishnan proposed openstack/nova: Added CLI args for nova-policy check command  https://review.openstack.org/38291512:41
*** jaosorior has quit IRC12:41
sdaguesahid: can you pull that comment outside of the parameter list and put it before the function? That's really odd style as it is, and makes it a little hard to realize all those parameters are part of the same function12:41
sdaguethe fix seems fine otherwise12:42
*** omnipresent has quit IRC12:42
sahidsdague: yes sure, let me do that and thanks to have reviewed it12:42
*** zenoway has joined #openstack-nova12:42
garykmdbooth: no idea. i am trying to figure out where this is done…12:43
*** ducttape_ has joined #openstack-nova12:43
*** anush has joined #openstack-nova12:45
*** zenoway has quit IRC12:47
*** ducttape_ has quit IRC12:47
*** diana-test has joined #openstack-nova12:49
*** code-R_ has joined #openstack-nova12:49
*** links has quit IRC12:50
*** Pavo has quit IRC12:51
*** zenoway has joined #openstack-nova12:51
*** diana-test has quit IRC12:51
*** kaisers has quit IRC12:51
*** diana-test has joined #openstack-nova12:52
*** zenoway has quit IRC12:52
*** zenoway has joined #openstack-nova12:52
*** code-R has quit IRC12:53
openstackgerritsahid proposed openstack/nova: libvirt: fix DiskSmallerThanImage when block migrate ephemerals  https://review.openstack.org/37856012:53
*** diana-test has quit IRC12:55
*** Pavo has joined #openstack-nova12:55
*** liverpooler has quit IRC12:56
*** diana-test has joined #openstack-nova12:56
*** anush has quit IRC12:56
*** diana-test has quit IRC12:56
*** ociuhandu has joined #openstack-nova12:56
*** zenoway has quit IRC12:57
*** burgerk has joined #openstack-nova12:57
*** ociuhandu has quit IRC12:59
*** garyk has quit IRC13:02
*** jheroux has joined #openstack-nova13:02
*** thorst_ has joined #openstack-nova13:02
*** alexchadin has quit IRC13:02
*** ociuhandu has joined #openstack-nova13:04
*** dave-mccowan has joined #openstack-nova13:04
*** mdrabe has joined #openstack-nova13:04
*** ayoung has joined #openstack-nova13:04
*** moshele has quit IRC13:04
*** moshele has joined #openstack-nova13:05
*** links has joined #openstack-nova13:05
openstackgerritTakashi NATSUME proposed openstack/nova-specs: Enable cold migration with target host  https://review.openstack.org/33428613:05
*** namnh has joined #openstack-nova13:05
*** ducttape_ has joined #openstack-nova13:06
*** vishwanathj has joined #openstack-nova13:07
*** tbachman has joined #openstack-nova13:07
*** agireud has quit IRC13:10
*** tongli has joined #openstack-nova13:11
*** whenry has joined #openstack-nova13:11
*** roeyc has quit IRC13:11
namnhHi everyone, I am trying to upgrade Nova from Mitaka to Newton, I finished all step but in the final step: (nova-manage db online_data_migrations). It had error like this: ERROR nova.db.sqlalchemy.api [req-8328271d-7d1c-4aca-bbcd-5651cccba6ca None None] Data migrations for PciDevice are not safe, likely becausest version . do you encounter this bug?13:13
*** Dinesh_Bhor has quit IRC13:13
*** gcb has quit IRC13:15
*** hferenc_ has joined #openstack-nova13:16
*** hemanthm|away is now known as hemanthm13:16
*** vladikr has joined #openstack-nova13:16
*** mriedem has joined #openstack-nova13:16
*** roeyc has joined #openstack-nova13:17
*** agireud has joined #openstack-nova13:18
namnhavolkov, Hi Mr.Volkow, are you free?13:19
avolkovnamnh: hi, yes13:20
edleafenamnh: Yes, that was reported yesterday: https://bugs.launchpad.net/nova/+bug/163073213:21
openstackLaunchpad bug 1630732 in OpenStack Compute (nova) newton "online_data_migrations does not ignore deleted services" [High,In progress] - Assigned to Dan Smith (danms)13:21
namnhavolkov, thanks for your reply. :) Could you please see  above my message.13:22
namnhavolkov, I saw a patch set which you were co-author: https://review.openstack.org/#/c/290793/13:23
openstackgerritDrew Thorstensen (thorst) proposed openstack/nova-specs: PowerVM Initial Driver Work  https://review.openstack.org/38177213:23
namnhavolkov, oh. you mean that, currenlty this is a bug13:23
*** prateek has quit IRC13:25
*** prateek has joined #openstack-nova13:25
*** moshele has quit IRC13:26
*** mkoderer has joined #openstack-nova13:27
*** ducttape_ has quit IRC13:28
*** haplo37_ has quit IRC13:28
*** riccardo_ has quit IRC13:29
*** zul has quit IRC13:30
*** haplo37_ has joined #openstack-nova13:30
*** ijw has joined #openstack-nova13:31
markus_zmriedem: Would you remove the -2 from https://review.openstack.org/#/c/323765/ please? I'm moving the spec to Ocata with https://review.openstack.org/#/c/382526/13:31
*** baoli_ has joined #openstack-nova13:32
mriedemmarkus_z: i will after the bp is approved13:33
*** zul has joined #openstack-nova13:33
openstackgerritJulian proposed openstack/nova: Transform instance.power_off notifications  https://review.openstack.org/33076113:34
mriedemmarkus_z: commented on the spec patch13:34
markus_zIt was approved for Newton but got blocked due to a qemu bug. The qemu bug is fixed (I'm currently testing that).13:34
openstackgerritJulian proposed openstack/nova: Transform instance.snapshot notifications  https://review.openstack.org/33012813:34
*** baoli has quit IRC13:35
openstackgerritMaciej Szankin proposed openstack/nova: config options: improve libvirt imagebackend  https://review.openstack.org/37850713:35
markus_zthanks for the link to the process, will do13:35
*** bnemec has joined #openstack-nova13:36
openstackgerritMaciej Szankin proposed openstack/nova: Improve help text for network  options.  https://review.openstack.org/34120313:36
*** ijw has quit IRC13:37
*** vladikr has quit IRC13:38
*** vladikr has joined #openstack-nova13:38
openstackgerritMaciej Szankin proposed openstack/nova: Transform instance.reboot notifications  https://review.openstack.org/38295913:41
*** prateek_ has joined #openstack-nova13:41
*** mlavalle has joined #openstack-nova13:42
*** prateek has quit IRC13:43
*** burgerk has quit IRC13:45
*** thorst__ has joined #openstack-nova13:46
*** eharney has joined #openstack-nova13:47
*** roeyc has quit IRC13:48
*** akshai has joined #openstack-nova13:49
*** claudiub has joined #openstack-nova13:49
*** thorst_ has quit IRC13:49
openstackgerritMarkus Zoeller (markus_z) proposed openstack/nova-specs: Libvirt: Use the virtlogd deamon  https://review.openstack.org/38252613:50
openstackgerritTakashi NATSUME proposed openstack/nova-specs: Check the destination host in cold migration  https://review.openstack.org/33472513:50
*** cmartin has joined #openstack-nova13:50
*** akshai_ has joined #openstack-nova13:50
*** hshiina has quit IRC13:51
mriedemnova meetingathon in 7 minutes13:52
*** GB21 has quit IRC13:53
claudiubdanpb: hello. if you have some spare time, can you take a look at the hyper-v ovs vif plug thingy? Thanks! https://review.openstack.org/#/c/140045/13:54
*** akshai has quit IRC13:54
*** mtanino has joined #openstack-nova13:56
openstackgerritStephen Finucane proposed openstack/nova: live-mig: Add claims and proper resource tracking  https://review.openstack.org/24448913:57
openstackgerritStephen Finucane proposed openstack/nova: libvirt: pass libvirt reference methods to migration  https://review.openstack.org/32899713:57
openstackgerritStephen Finucane proposed openstack/nova: libvirt: live-migrate updates NUMA and cpus in the XML  https://review.openstack.org/28674413:57
jrollmriedem: some folks are interested in a x-p meetup with nova on line 66 here https://etherpad.openstack.org/p/ironic-ocata-summit13:57
jrollthink we can get some of y'all on wednesday evening when there's no sessions?13:57
jrollno nova sessions*13:57
*** ducttape_ has joined #openstack-nova13:58
mriedemjroll: we have a 5pm wed nova/neutron session in the neutron room13:58
mriedemis there a slot before that?13:59
jrollmriedem: we have a 5:55-6:3513:59
dansmithoof13:59
jrollikr13:59
*** thorst__ is now known as thorst_13:59
jrollit affects api and probably placement so I expect everybody will have opinions13:59
*** tonytan4ever has joined #openstack-nova13:59
mriedem6pm is tapas time13:59
jrollnot for me :(13:59
*** sdake has joined #openstack-nova13:59
jrollI'll bug you in the meeting and see what we can figure out14:00
mriedemok14:00
mriedemif it's placement related i'd want dansmith and jaypipes there14:00
*** anush has joined #openstack-nova14:01
*** bvanhav has quit IRC14:01
*** bvanhav has joined #openstack-nova14:03
mriedemmdbooth: when do you head out from the summit?14:04
mdboothmriedem: I saw that comment. Friday mid-afternoon.14:04
mdboothThat slot is fine with me.14:04
mriedem11am is ok? good.14:04
mriedemif that doesn't work we can move it14:04
namnhedleafe, Wow, thanks for your information, it's really useful to me. Now I have to go out my office. Good bye.14:05
mdboothmriedem: Nope, that's fine.14:05
*** namnh has quit IRC14:06
*** tonytan4ever has quit IRC14:06
openstackgerritAndrew Laski proposed openstack/nova-specs: Proposal for cells v2 scheduling interaction  https://review.openstack.org/38127514:06
*** tonytan4ever has joined #openstack-nova14:06
*** bvanhav_ has joined #openstack-nova14:07
openstackgerritLee Yarwood proposed openstack/nova: Do not reserve device before reserving the volume  https://review.openstack.org/29079314:07
*** READ10 has joined #openstack-nova14:08
*** bvanhav has quit IRC14:08
openstackgerritsahid proposed openstack/nova: virt: remove small pages request  https://review.openstack.org/17207914:09
*** anush has quit IRC14:10
*** zhenguo has joined #openstack-nova14:12
*** burgerk has joined #openstack-nova14:13
*** kaisers has joined #openstack-nova14:14
*** prometheanfire has left #openstack-nova14:14
*** sdake has quit IRC14:15
*** rama_y has joined #openstack-nova14:15
*** pbandark has quit IRC14:17
*** bvanhav_ is now known as bvanhav14:18
*** moshele has joined #openstack-nova14:19
*** jamesdenton has joined #openstack-nova14:22
*** parora has joined #openstack-nova14:22
*** liverpooler has joined #openstack-nova14:23
*** zenoway has joined #openstack-nova14:24
*** jlanoux has quit IRC14:25
*** prateek_ has quit IRC14:25
openstackgerritMaciej Szankin proposed openstack/nova: Remove config option snapshot_name_template  https://review.openstack.org/37120314:27
openstackgerritDrew Thorstensen (thorst) proposed openstack/nova-specs: PowerVM Initial Driver Work  https://review.openstack.org/38177214:28
*** zenoway has quit IRC14:28
*** gouthamr has quit IRC14:29
*** mgoddard_ has joined #openstack-nova14:29
*** gouthamr has joined #openstack-nova14:31
*** ekuris has quit IRC14:31
*** mgoddard has quit IRC14:31
*** ijw has joined #openstack-nova14:33
*** tanvirt has joined #openstack-nova14:34
sdaguedansmith: http://logs.openstack.org/60/378560/4/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/b18e638/logs/devstacklog.txt.gz#_2016-10-06_13_19_15_279 - is kind of odd14:35
*** READ10 has quit IRC14:35
sdaguethat seems to be happening in regular gate runs14:35
dansmithsdague: that's regular db sync complaining about the api db?14:36
alaskilooks like that14:36
*** anush has joined #openstack-nova14:36
dansmithsdague: I wonder if some change recently in a decorator or something is initializing the api db connection early or something14:37
dansmithbut it also looks unfatal?14:38
dansmithreally strange14:38
*** ijw has quit IRC14:38
sdaguedansmith: it's unfatal14:39
sdaguebut ugly14:39
sdagueand scary14:39
dansmithsdague: I know, but I'm trying to figure out why it would be doing that at all, and why it does't seem to care that it failed14:39
*** zenoway has joined #openstack-nova14:40
*** Hosam has joined #openstack-nova14:40
dansmithI imagine this is the thread to follow:" ERROR: could not access cell mapping database - has api db been created?14:40
*** Hosam_ has joined #openstack-nova14:40
*** tanvirt has quit IRC14:41
openstackgerritJohn Garbutt proposed openstack/nova-specs: WIP: API: rework security group APIs  https://review.openstack.org/38241414:41
dansmithah yeah14:41
alaskiit's looking up a cellmapping14:41
alaskisince syncing cell0 was added14:41
*** sridharg has quit IRC14:41
*** annegentle has joined #openstack-nova14:42
alaskiand the nova_api db is created by devstack right after running the cell db sync14:42
*** anush has quit IRC14:43
dansmithalaski: so with that ordering we're just ignoring the mapping and going straight to the config'd db I guess?14:43
dansmithnot sure I see why we're getting a stack trace though14:44
alaskiyes. thge cell0 db never gets synced14:44
mriedemdanpb: can you jump into the nova meeting?14:44
*** Hosam has quit IRC14:44
danpbwhat channel ?14:44
mriedem#openstack-meeting14:45
dansmithalaski: I guess the decorator is logging that traceback for us14:45
openstackgerritAugustina Ragwitz proposed openstack/nova-specs: Add spec for lower case metadata keys only  https://review.openstack.org/35084314:45
auggysdague: thanks for your input on the lc-metadata spec, i've just pushed an update to address some of the comments14:45
auggyand to give mtreinish credit for his help ;)14:45
sdagueauggy: great, thanks14:45
alaskidansmith: the db context manager?14:45
dansmithalaski: yeah14:46
*** annegentle has quit IRC14:47
mtreinishauggy: hah, it's just a single patch. You didn't have to rev the spec for that14:47
dansmithalaski: so I guess we need to reverse those orderings.. we could run it again after api, but we won't be able to avoid that error log I suppose14:47
*** kaisers has quit IRC14:47
auggymtreinish: :P it was a big help14:48
alaskidansmith: we would need to sync the api db first as well or it will fail on the table lookup14:48
dansmithalaski: yeah I meant sync cell0 after we're done with the api, but...14:49
dansmithalaski: we can just reverse the ordering and do api first I assume right?14:49
alaskidansmith: the enginefacade code is horrible to try to trace, but I'm not seeing where it would log the exception yet14:49
dansmithalaski: just in the cell mapping lookup14:49
*** lpetrut has quit IRC14:49
dansmithalaski: I know, the traceback is useless and I don't know why14:49
*** rrecio has joined #openstack-nova14:49
alaskidansmith: creating and syncing the api db first should be fine14:50
dansmithalaski: I think we never even run our db code, because when we enter the decorator it tries to connect and start a transaction against api_db, fails, and we never run our code14:50
dansmithalaski: okay14:50
*** Pavo has quit IRC14:51
dansmithalaski: sdague I'd like to make this less of an explosion for people that happen to reverse these steps as well, but I'm not sure there is a practical way to do that because of how enginefacade works14:51
openstackgerritGábor Antal proposed openstack/nova: Removed unnecessary parantheses in return statements  https://review.openstack.org/37054914:51
sdaguedansmith: that makes me kind of a sad panda if that's true14:51
alaskiahh, I found the log line. still not sure how it gets hit yet14:52
sdaguedansmith: I guess we could register a high priority bug against oslo.db for that and see if that team can take a look?14:52
*** rrecio_ has joined #openstack-nova14:52
alaskihttp://git.openstack.org/cgit/openstack/oslo.db/tree/oslo_db/sqlalchemy/exc_filters.py#n37314:53
dansmithsdague: it's kinda fundamental14:53
dansmithsdague: unless we decorate one call with a "don't log errors please" thing and just handle this path because of how we use it14:53
*** annegentle has joined #openstack-nova14:53
alaskiit's also due to our attempt to implicitly do the right thing here without asking much of users14:54
*** erlon has quit IRC14:54
dansmithsdague: we could try to connect to api_db ourselves quickly to validate that it's been setup and then not call this if we think it's going to fail14:54
*** ildikov has quit IRC14:54
alaskiwe're really calling the sync method incorrectly since the db isn't setup14:54
dansmithsdague: but that's kinda going round14:54
*** rrecio has quit IRC14:54
*** armax has joined #openstack-nova14:54
dansmithalaski: which is why hiding the error here doesn't make sense normally, only here because we're in manage14:54
alaskiyeah. I agree with that14:55
*** DuncanT has quit IRC14:55
sdagueright, I think this is one of the cases where the error handling all being buried makes for bad ops UX, and in this specific case we should be able to not stack trace14:55
*** mkoderer has quit IRC14:55
*** geekinutah has quit IRC14:55
*** anush has joined #openstack-nova14:55
dansmithsdague: well, it probably makes for good ops UX normally, just not right here14:55
sdaguedansmith: yeh, is there any existing way to disable logging for calls?14:55
*** jlanoux has joined #openstack-nova14:55
dansmithsdague: this is a special case14:55
*** csatari has quit IRC14:55
*** macsz has quit IRC14:55
*** boris-42 has quit IRC14:55
*** dougwig has quit IRC14:55
*** cargonza has quit IRC14:55
*** ediardo has quit IRC14:55
*** andrewbogott has quit IRC14:55
*** coolsvap has quit IRC14:55
*** patrickeast has quit IRC14:55
sdaguedansmith: yeh14:55
*** tomhambleton has quit IRC14:55
*** fyxim has quit IRC14:55
*** Pavo has joined #openstack-nova14:55
dansmithsdague: well, that's what I mean.. we *could* ask them for a special flag, it just seems kinda like a weird request14:56
dansmitha "disable logging in this one method" flag14:56
*** hieulq__ has joined #openstack-nova14:56
dansmithbut then again, it would disable logging for this path at runtime, which we don't want14:56
dansmithunless we have some way to toggle14:56
dansmithI dunno14:56
*** tanvirt has joined #openstack-nova14:56
dansmithI have to be on a call in four minutes14:57
johnthetubaguymriedem: I was planning on attending the PTG14:57
mriedemok14:57
sdaguedansmith: yeh, it seems like we honestly would like to have a context manager that shushs the logging during a block where we call this14:58
dansmithmriedem: I expect I will be there14:58
sdagueand returns to normal outside of it14:58
sdaguemriedem: I expect to be at the PTG14:58
dansmithsdague: yeah, but.. I guess..14:58
dansmithsdague: we could just disable error logging before this call and restore the level after :)14:58
sdaguedansmith: we could14:58
*** haplo37 has quit IRC14:58
*** kaisers has joined #openstack-nova14:59
sdagueok, how about I file a bug, and we try to see what oslo.db folks have to say before completely working around it on the nova side :)14:59
*** salv-orlando has joined #openstack-nova14:59
*** anush has quit IRC14:59
*** anush_ is now known as anush15:00
dansmithsdague: sure, you do that :)15:00
*** anush_ has joined #openstack-nova15:00
*** takashin has left #openstack-nova15:00
alaskimriedem: I expect I'll be at the PTG. I can actually drive to this one15:00
*** haplo37 has joined #openstack-nova15:01
mriedemhferenc: so i think the image meta / flavor extra specs format thing is a known issue15:01
mriedemhferenc: as i said in the meeting, melwitt recently triaged a similar bug that turned out to be a problem in glance15:01
mriedemhferenc: so she probably has more details15:01
edleafejaypipes: dansmith: bauzas: quick reminder that you need to update the etherpad with the long-term objective and path to get there for the placement DB: https://etherpad.openstack.org/p/placement-optional-db-spec15:01
sfinucanmriedem: hferenc I don't think the issue melwitt was looking at was quite the same15:02
melwittI was looking at a mismatch between what nova uses and what glance uses for certain image meta keys15:03
*** mjura has quit IRC15:03
melwitthttps://bugs.launchpad.net/glance/+bug/156819115:03
openstackLaunchpad bug 1568191 in Glance "Pre-defined VirtCPUTopology metadata should have the same name as flavor extra specs in nova" [Medium,In progress] - Assigned to Ying Zuo (yingzuo)15:03
*** bdeschenes has joined #openstack-nova15:03
hferenc_i see15:04
kashyapmdbooth: Not sure if you're following that upstream thread (both on openstack-dev & libvir-list), Eric made interesting replies, and even responds to your concern of "Why should libvirt munge the reporting"15:04
mriedemhferenc: as far as i can tell, the image metadata format is os_foo_bar15:05
mriedemwith the os_ prefix15:05
sdaguehttps://bugs.launchpad.net/oslo.db/+bug/163103315:05
openstackLaunchpad bug 1631033 in oslo.db "scary traceback if nova-manage db sync is run before nova-manage api_db sync" [Undecided,New]15:05
mriedemflavor extra specs is os:foo_bar15:05
mriedemwith the os: prefix15:05
mdboothkashyap: Yes, I saw that. I don't disagree with him as I said earlier, but it's messy.15:05
mriedeme.g. https://review.openstack.org/#/c/209581/49/releasenotes/notes/hyperv-uefi-secure-boot-a2a617ac2c313afd.yaml15:05
mdbooth'uncomfortable' rather than 'opposed'15:05
*** oanson has quit IRC15:05
mdboothStill better to use the other api, I think15:05
*** diogogmt has joined #openstack-nova15:06
hferenc_mriedem yeah, the strange thing is that horizon uses os_foo_bar even for flavor extra specs15:06
openstackgerritTakashi NATSUME proposed openstack/nova: Stop using mox stubs in nova/tests/unit/cells  https://review.openstack.org/32328715:08
openstackgerritTakashi NATSUME proposed openstack/nova: Remove mox in test_block_device_mapping_v1.py  https://review.openstack.org/30874615:08
melwittI think that's okay as long as the key (after the prefix) matches what's in nova. because IIRC nova strips the prefixes off before accessing keys15:08
*** marst has quit IRC15:08
openstackgerritTakashi NATSUME proposed openstack/nova: api-ref: Fix wrong parameters in os-volumes.inc  https://review.openstack.org/34711215:08
openstackgerritTakashi NATSUME proposed openstack/nova: Add a hacking rule for string interpolation at logging  https://review.openstack.org/33484415:09
*** jlanoux_ has joined #openstack-nova15:09
*** annegentle has quit IRC15:09
*** jlanoux has quit IRC15:10
openstackgerritRadoslav Gerganov proposed openstack/nova: Fix API doc for os-console-auth-tokens  https://review.openstack.org/34096915:10
*** armax has quit IRC15:10
kashyapmdbooth: What do is the "other" API?15:10
kashyaps/do//15:11
hferenc_i'd not agree, because in my experience the vm fails to boot if i specify flavor metadata using the os_foo_bar format15:11
*** armax has joined #openstack-nova15:11
*** fyxim has joined #openstack-nova15:11
hferenc_see https://bugs.launchpad.net/nova/+bug/1582693 for different usecases15:12
openstackLaunchpad bug 1582693 in OpenStack Compute (nova) "Image and flavor metadata for libvirt watchdog is handled erroneously" [Undecided,In progress] - Assigned to Richil Bhalerao (richil-bhalerao)15:12
cmartinHey I've had a bugfix up for a while now on resizing vm's and would really appreciate some reviews https://review.openstack.org/#/c/370861/15:12
mriedemhferenc: seems like a horizon bug15:13
mriedemhferenc: i just left a comment in the bug15:13
*** Hosam_ has quit IRC15:13
*** Hosam has joined #openstack-nova15:14
*** lucasagomes is now known as lucas-hungry15:15
mriedemhferenc: the docs also use <capability>: for extra_specs http://docs.openstack.org/admin-guide/compute-flavors.html#extra-specs15:15
mriedemi'm not really sure why nova doesn't start enforcing that behavior though with a microversoin15:15
mriedemso that flavor extra specs require a <capability>: prefix15:16
mriedemdanpb: dansmith: jaypipes: sdague: alaski: johnthetubaguy: ^ any reason we couldn't/shouldn't do that?15:16
johnthetubaguymriedem: I was hoping we would go that way in a microversion, unify the image and extra specs stuff using the capabilities/traits stuff15:17
*** r-daneel has joined #openstack-nova15:17
mriedemwe don't control image meta anymore15:18
*** code-R_ has quit IRC15:18
mriedemwell, after alex_xu's bp is done to 404 on image meta proxy15:18
alaskiit's a little odd because there are many extra specs already out there that don't follow that pattern. so we'll force deployments to switch between microversions to set what they need15:18
melwitthferenc_: oh, I see, so horizon is using the glance prefixed name on a nova flavor extra spec, which is a problem15:18
mriedemit's certainly easier to validate the flavor extra specs format given the :15:18
*** READ10 has joined #openstack-nova15:18
johnthetubaguymriedem: I am on about what we interpret in provided image metadata15:18
danpbmriedem: sorr, do what ?15:18
melwittit has to be glance prefix + image meta or nova prefix + flavor extra spec15:18
*** dougwig has joined #openstack-nova15:19
*** marst has joined #openstack-nova15:19
*** Hosam has quit IRC15:19
mriedemalaski: nova under the covers could still handle flavor extra specs with both formats,15:19
alaskiI would think a better path would be to put these new capabilities things into something different in the api15:19
mriedembut with a microversion we could start enforcing the new format15:19
johnthetubaguyyeah, need to handle the old ones, and map it across if possible I guess15:20
sdaguemriedem: so, the idea being that we filter the setting of specs to only allow setting the namespaced versions with a new microversion?15:20
danpbmriedem: we can't use underscore syntax in flavour extra specs without clashing with other code15:20
openstackgerritGábor Antal proposed openstack/nova: Removed unnecessary parantheses in return statements  https://review.openstack.org/37054915:20
johnthetubaguyI guess we are saying, drop extra specs and add a new thing?15:20
danpbone of the schedular filters matches on everything that doesn't have a  xxx:  prefix15:20
mriedemsdague: yes15:20
alaskijohnthetubaguy: we would be implicitly saying extra specs are a new thing now15:20
sdaguemriedem: that seems fine, then some legacy mapping for things we know15:20
*** mkoderer has joined #openstack-nova15:21
sdaguemriedem: given the degree of confusion here, probably worth writing a quick spec on it to make sure we're all clear on that15:21
*** kfarr has joined #openstack-nova15:21
sdaguedanpb: it's actually the opposite idea, make sure everything comes through with a :15:21
sdagueso that we don't get people confusedly pushing _ versions of things and being confused they don't work15:22
sdaguewhich is what has happened in horizon apparently15:22
danpbwhat would actually be nice is if we actually had an object defining extra specs like we did for image meta, so we could actually validate them sensibly15:22
mriedemthe aggregate_image_properties_isolation_separator config option kind of breaks this15:23
danpbas even if you have the right prefix  : vs _, doesn't mean you've got the data right, or that this version of nova supports it15:23
sdaguemriedem: well, start with deprecating that15:23
* hferenc_ is sorry for starting a tsunami15:24
*** boris-42 has joined #openstack-nova15:24
*** mdrabe has quit IRC15:24
sdaguedanpb: given that the field is currently free form, how would that work with other services / users setting arbitrary things?15:24
alaskimy only issue is that both extra specs and scheduler filters are open for use/extension and some deployers have taken advantage of that to do their own things. This would be changing things up on them. But I guess as long as the prefix isn't constrained it's just a small change for those deployers.15:24
alaskiso I think I've convinced myself there's no reason not to do it15:24
mriedemyeah AggregateInstanceExtraSpecsFilter expects the flavor extra specs to have the : format15:24
sdaguealaski: is it only deployers changing this because of scheduler filters?15:25
*** diga has quit IRC15:25
sdagueare there any ways in which other services use this data?15:25
*** zenoway has quit IRC15:25
alaskisdague: no. I did some stuff with policy using extra specs as well15:25
alaskigut if the only requirement is that it now needs a foo: appended to it I think it'll be a small change for folks15:26
*** zenoway has joined #openstack-nova15:26
alaskis/gut/but/15:26
mriedemalaski: as far as i can tell, AggregateInstanceExtraSpecsFilter and ComputeCapabilitiesFilter both require the extra spec to have a <cap>: prefix15:26
mriedemelse they don't work15:26
mriedemti's hard-coded15:26
sdaguealaski: I wonder if making a registry in nova, which includes an entrypoint where you can, as a deployer, register your own things for validation would be ok15:26
mriedemAggregateImagePropertiesIsolation is configurable15:26
alaskimriedem: it's not required that those are used though15:26
auggymriedem: i'm planning to crash the PTG and contribute many bad jokes15:26
mriedemalaski: true15:26
sdaguejust looking for a : helps with one class of errors15:26
sdaguebut actually validating properties against a known list, like danpb suggested, catches a whole lot more15:27
mriedemso we can't enforce : in flavor extra specs b/c of out of tree scheduler filters?15:27
alaskisdague: this is why I suggested we add a new API thing that's not flavor extra specs to set these things15:27
mriedemi'd argue that any out of tree scheduler filter should support :15:27
mriedemin addition to whatever other format15:27
sdaguemriedem: sure, I think that's fine15:27
sdagueI was just thinking about the hard validation15:28
sdaguewhich is probably also fine, as long as there is an easy mechanism for people to add register their things15:28
sdaguealaski: so... you are thinking flavor extra specs registration API?15:28
*** hieulq__ has quit IRC15:28
sdagueor some other thing entirely?15:29
alaskiwell, I would stop calling them extra_specs. even if they're stored that way on flavors15:29
sdaguealaski: sure15:30
alaskibut yes, some other thing with strict validation on what's added15:30
sdagueI guess the question is, if we have hard strict validation, how do you whitelist in new values15:30
sdagueand it could be an API call15:30
sdagueor it could be an entry point with definitions15:30
sdagueor something else15:30
*** zenoway has quit IRC15:31
*** bvanhav has quit IRC15:31
alaskido we need to whitelist in values at that point? extra specs would still be open and free form15:31
*** mdrabe has joined #openstack-nova15:31
sdaguealaski: if we wanted hard strict validation, yes15:31
*** bvanhav has joined #openstack-nova15:31
sdagueto defend against15:32
sdaguequota:cpu_peroid15:32
alaskiokay. I thought we had a definitive list like I see at http://docs.openstack.org/admin-guide/compute-flavors.html#extra-specs15:32
*** tonytan_brb has joined #openstack-nova15:32
danpbalaski: lol, no that docs is just someones best guess as to what we use15:32
alaskiif we still need an extension mechanism then keeping it in the extra specs call seems better to me15:32
alaskidanpb: gotcha15:32
danpbuntil we have a formal versioned object defining this, there is no definitive list15:33
danpbit took us ages to track down all the secret image properties nova code was relying on15:33
*** geekinutah has joined #openstack-nova15:33
danpband flavour extra specs is an even bigger mess than image properties was15:33
alaskiit seems like that's the starting point to this then. Have a definitive list we can use to guard with15:34
*** ijw has joined #openstack-nova15:34
*** suresh12 has joined #openstack-nova15:34
*** peter-hamilton has quit IRC15:34
*** tonytan4ever has quit IRC15:35
*** Matti_ has joined #openstack-nova15:35
*** andrewbogott has joined #openstack-nova15:36
*** erlon has joined #openstack-nova15:36
alaskimriedem: back to your original question, restricting on ':' seems fine to me. A longer term plan may be to add a new strictly validating api call if we have a way to do that strict validation.15:36
*** DuncanT has joined #openstack-nova15:36
*** ducttape_ has quit IRC15:37
*** andrewbogott has quit IRC15:37
*** andrewbogott has joined #openstack-nova15:37
*** patrickeast has joined #openstack-nova15:37
*** ildikov has joined #openstack-nova15:38
*** andreas_s has quit IRC15:38
*** suresh12 has quit IRC15:39
*** ijw has quit IRC15:40
*** cargonza has joined #openstack-nova15:41
*** macsz has joined #openstack-nova15:41
*** moshele has quit IRC15:42
mriedemalaski: sdague: ok i'll give myself a todo to try and start a spec on the : validation for extra specs, seems like a no-brainer15:42
*** ediardo has joined #openstack-nova15:43
mriedemhferenc: ^15:43
mriedemhferenc: i'd probably take that to the horizon team though that flavor extra specs are assumed to be prefixed with the :15:43
mriedemand they should start doing that15:43
hferenc_i think i managed to keep up with you guys15:44
hferenc_and since is started all this please let me know if i can help15:44
hferenc_*i15:45
*** csatari has joined #openstack-nova15:46
mriedemi need to fire up a horizon and see what it shows15:46
*** tomhambleton has joined #openstack-nova15:46
*** coolsvap has joined #openstack-nova15:47
*** paul-carlton2 has quit IRC15:47
*** bvanhav has quit IRC15:48
*** ftersin has left #openstack-nova15:50
*** ducttape_ has joined #openstack-nova15:52
*** diogogmt_ has joined #openstack-nova15:53
*** diogogmt has quit IRC15:53
*** diogogmt_ is now known as diogogmt15:53
*** kaisers has quit IRC15:53
*** akshai_ has quit IRC15:55
*** haplo37_ has quit IRC15:57
*** haplo37_ has joined #openstack-nova15:59
*** rcernin has quit IRC16:01
*** mdrabe has quit IRC16:01
*** mdrabe has joined #openstack-nova16:01
*** david-lyle has joined #openstack-nova16:01
*** jheroux has quit IRC16:02
*** bvanhav has joined #openstack-nova16:02
*** avolkov has quit IRC16:03
openstackgerritJABAR ALI proposed openstack/python-novaclient: commit cc963fcdde86c6b5724bd8fa80cd4a2eb7d4b02c Author: Jabar Ali <jabbar.ali@gmail.com> Date:   Thu Oct 6 16:59:37 2016 +0530  https://review.openstack.org/38286416:05
*** kberger has quit IRC16:05
*** kberger has joined #openstack-nova16:05
*** jheroux has joined #openstack-nova16:06
*** avolkov has joined #openstack-nova16:08
*** avolkov` has joined #openstack-nova16:09
*** nkrinner is now known as nkrinner_afk16:09
*** zenoway has joined #openstack-nova16:09
*** akshai has joined #openstack-nova16:11
*** lucas-hungry is now known as lucasagomes16:12
*** zenoway has quit IRC16:14
*** jistr is now known as jistr|afk16:14
vladikrI've written some functional tests to cover the VF/PF passthrough that we've submitted a while back. I'd be happy to get more reviews please :) : https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:master+topic:sriov_numa_func_test16:14
*** raunak has joined #openstack-nova16:16
*** marst has quit IRC16:20
*** marst has joined #openstack-nova16:22
*** penick has joined #openstack-nova16:24
*** nic has joined #openstack-nova16:25
*** markus_z has quit IRC16:26
*** jlanoux_ has quit IRC16:26
*** annegentle has joined #openstack-nova16:27
*** mvk has quit IRC16:27
*** Swami has joined #openstack-nova16:28
*** ducttape_ has quit IRC16:30
*** danpb has quit IRC16:30
*** tesseract- has quit IRC16:31
*** bvanhav has quit IRC16:31
*** xyang1 has joined #openstack-nova16:32
*** zhenguo has quit IRC16:33
*** mat128 is now known as mat128|lunch16:34
*** ijw has joined #openstack-nova16:36
mriedemanyone want to clean this up? https://review.openstack.org/#/c/345162/416:40
mriedemit's a duplicate original of the bug/patch that garyk reported which we talked about in the nova meeting16:40
*** ijw has quit IRC16:40
sfinucanmriedem: Isn't garyk going to work on that?16:42
mriedemsfinucan: on updating the original fix?16:42
mriedemidk16:42
mriedemhe's not around16:42
sfinucanmriedem: bdm isn't my area of expertise but I'll take a look16:43
mriedemit's just a matter of fixing the tests16:43
sfinucancool16:43
mriedemand probably adding a new test for the bug case16:43
sfinucanAlso, I'd like to help out on that extra_specs thing when it comes around. Had talked with johnthetubaguy about doing something there some time back16:43
mriedemok16:43
sfinucancool cool16:43
johnthetubaguyyeah, you had good insight into the NUMA things, so sounded like a good fit16:44
*** bvanhav has joined #openstack-nova16:47
*** sahid has quit IRC16:48
dansmithmriedem: so how is that happening?16:48
dansmithmriedem: no bdms for an instance..16:48
mriedemi felt like i'd actually triaged / fixed something like this before already b/c it happened in the gate16:49
mriedemit's a race between server create and the periodic task running i think16:49
dansmithI don't actually understand how bdms[uuid] works either16:49
dansmithmriedem: ah, that race makes sense, but the existing code doesn't16:50
dansmithbdms is a BlockDeviceMappingList, which is a .. list, indexed by integers, not uuids16:50
*** Pavo has quit IRC16:51
dansmithoh, that one is special16:51
mriedemthat's where you're wrong buddy16:51
mriedemyeah16:51
dansmithgotcha16:51
mriedemobj_make_dict_of_lists16:51
mriedemwah wah16:51
dansmithyeah16:51
mriedemso the guest is on the host16:51
dansmithsfinucan: you're doing this?16:51
mriedemthe instance is in the db16:51
mriedembut the bdm isn't16:51
dansmithaye16:52
dansmithsfinucan: otherwise I'll be glad to16:52
sfinucandansmith: It's 5:50pm, so it's all yours :)16:52
dansmithsfinucan: ack16:52
mriedemoh come on16:52
mriedem6pm psh16:52
dansmithmriedem: sfinucan is a lightweight16:53
mriedemi guess16:53
mriedemi'm particularly offended because i know he's young16:53
dansmithmriedem: also, he has a government-mandated pub commitment on the horizon16:53
mriedemoh yeah i guess that's true16:53
mriedemgotta get that 16oz in16:53
dansmithright16:53
mriedemor else the queen will nag you16:53
dansmithdon't want him getting in trouble16:53
sfinucansfinucan has an Ireland match to watch16:53
sfinucanand maybe a sup of beer...16:54
*** tonytan_brb is now known as tonytan4ever16:54
*** ducttape_ has joined #openstack-nova16:54
*** Apoorva has joined #openstack-nova16:54
*** Pavo has joined #openstack-nova16:55
*** derekh has quit IRC16:56
*** jistr|afk is now known as jistr16:57
*** ralonsoh has quit IRC16:59
*** Apoorva_ has joined #openstack-nova17:01
*** yamahata has quit IRC17:01
*** rama_y has quit IRC17:01
*** sdake has joined #openstack-nova17:01
*** yamahata has joined #openstack-nova17:01
*** slaweq_ has joined #openstack-nova17:01
*** jamielennox|away has quit IRC17:02
*** david-lyle has quit IRC17:03
cfriesen_sfinucan: just reviewed https://review.openstack.org/#/c/244489...we need to sort out the async setting of migration status to "finished" vs "completed"17:04
*** Apoorva has quit IRC17:04
*** CustosLimen has left #openstack-nova17:05
*** woodster_ has joined #openstack-nova17:05
*** Apoorva_ has quit IRC17:05
*** unicell has quit IRC17:07
*** mvk has joined #openstack-nova17:10
*** Apoorva has joined #openstack-nova17:12
*** jamielennox|away has joined #openstack-nova17:16
*** jamielennox|away is now known as jamielennox17:16
*** slaweq_ has quit IRC17:21
*** TravT_ has joined #openstack-nova17:22
*** slaweq_ has joined #openstack-nova17:23
*** TravT_ has quit IRC17:24
*** zz_dimtruck is now known as dimtruck17:24
*** bnemec has quit IRC17:25
*** david-lyle has joined #openstack-nova17:25
*** TravT has quit IRC17:25
openstackgerritSTEW TY proposed openstack/nova: Transform instance.unshelve notifications  https://review.openstack.org/38256517:27
*** jvgrant_ has quit IRC17:28
mriedemhferenc: so i was playing around with flavor extra specs in horizon, and it looks like the flavor extra specs you can choose come from the glance metadata catalog17:29
mriedemand some of the items have the namespace: prefix and some have namespace_17:29
dianadansmith: For simple tenant usage paging, were you thinking of paging the tenant list (tenant_usages) or the nested servers list (server_usages)? Current example response: http://paste.openstack.org/show/584548/17:29
mriedemhferenc: so it seems to be an issue in the glance metadata catalog?17:30
dansmithdiana: it's just one query to the db, right?17:30
dianadansmith: yup17:30
dansmithdiana: paginating that is the only thing that will help17:31
*** suresh12 has joined #openstack-nova17:31
*** unicell has joined #openstack-nova17:31
*** haplo37 has quit IRC17:32
*** slaweq_ has quit IRC17:33
*** annegentle has quit IRC17:34
*** tblakes has quit IRC17:34
*** haplo37 has joined #openstack-nova17:34
dianadansmith: But from an API response perspective are you expecting to page on the tenant_usages? Which won't help much in the "few tenats, many instances" case.17:35
*** slaweq_ has joined #openstack-nova17:35
dansmithdiana: I don't understand.. are you asking what the client should use as a marker?17:35
*** annegentle has joined #openstack-nova17:36
*** ijw has joined #openstack-nova17:36
*** sdague has quit IRC17:37
dianadansmith: Ignore the limit & marker concept for now. I'm asking at an API level. We currently return a list of tenant usages that each contain a list of server usages.17:37
*** hferenc_ has quit IRC17:37
dansmithum, I guess I don't know how we can ignore the limit/marker if we're talking about pagination.. we need to limit the results to N instances, which will be the total length of all server_usages combined, and which may end up with N tenants or one tenant, depending on the order in which they come out17:38
*** bnemec has joined #openstack-nova17:38
*** psachin has quit IRC17:39
mriedemhferenc: see https://bugs.launchpad.net/nova/+bug/1582693/comments/1617:39
openstackLaunchpad bug 1582693 in OpenStack Dashboard (Horizon) "Image and flavor metadata for libvirt watchdog is handled erroneously" [Undecided,New]17:39
mriedemhferenc: it's a glance metadefs bug i htink17:39
mriedem*think17:39
mriedemmelwitt: ^17:39
*** Margarita_S has quit IRC17:40
dianadansmith: I originally assumed you wanted the paging at the tenant_usages level. That is, you could limit it to just the first 5 tenants, with their nested server_usages (but those could still be really large lists).17:40
dansmithdiana: yeah that doesn't help17:40
dansmithand it's not how we're querying the db anyway, so it would be harder to do that I think, even if it made sense17:41
dianadansmith: right, because you can already do that with show filtered on tenant_id17:41
*** ijw has quit IRC17:41
*** lpetrut has joined #openstack-nova17:43
dansmithmriedem: the test that fails in that bdms patch is so effed17:44
mriedemlanguage please17:44
mriedemsorry,17:44
mriedemwatch your effing language please17:44
dianadansmith: If there are 2 tenats, each with 10 instances, and the page size is 5: what should the response look like? I think that answer will clarify intent for me.17:46
dansmithdiana: depends on the order that comes out of the db17:47
dansmithdiana: might be one tenant with five instances, two tenants with 2/3 instances, etc17:47
*** bnemec has quit IRC17:47
dianadansmith: But then the aggregate counts are off...17:47
*** roeyc has joined #openstack-nova17:48
dansmithdiana: you mean like the start/stop/total_hours etc?17:48
dianadansmith: the values in the tenants_usages dict are aggregates of the values in the servers_usages list17:49
*** artom has quit IRC17:49
dianadansmith: yes, the aggregate totals17:49
dansmithdiana: right, so this is pivoting from purely time-based to time-and-count based, so you'll have to correct those numbers, right?17:50
dianadansmith: those aggregates are supposed to reflect all the instances that are in that window for a given tenant though... correct? Not just some arbitrary group of them based on the order returned by the database.17:52
dansmithdiana: they're supposed to represent the instances in the server_usages list I would say17:52
dansmithdiana: obviously if you're a client and you're passing a limit, you're going to have to stitch things together from the pieces, just like any paginated result right?17:53
dianadansmith: so then horizon would need to stitch them all back together?17:53
diana(we typed at the same time, that wasn't in response)17:53
*** artom has joined #openstack-nova17:54
*** david-lyle has quit IRC17:55
dianadansmith: I'm starting to think that show should be paged instead of index then...17:55
dianadansmith: anyhoo, thanks for your time, I'll think on this some more17:55
*** hferenc has quit IRC17:56
*** bnemec has joined #openstack-nova17:57
*** Matti_ has quit IRC17:57
*** rama_y has joined #openstack-nova17:57
*** slaweq_ has quit IRC17:58
*** priteau has joined #openstack-nova18:01
*** tonytan4ever has quit IRC18:01
*** gszasz has quit IRC18:01
*** tonytan4ever has joined #openstack-nova18:01
*** dimtruck is now known as zz_dimtruck18:02
*** tanvirt has quit IRC18:02
*** ihrachys has quit IRC18:03
*** bnemec has quit IRC18:03
*** lpetrut has quit IRC18:05
*** lpetrut has joined #openstack-nova18:05
*** david-lyle has joined #openstack-nova18:06
*** cmartin has quit IRC18:06
jaypipeslbeliveau: reviewed.18:07
*** ldeptula has quit IRC18:08
*** cmartin has joined #openstack-nova18:08
*** bnemec has joined #openstack-nova18:08
*** annegentle has quit IRC18:08
lbeliveaujaypipes: thanks !  much appreciated18:09
jaypipeslbeliveau: thank *you* for the fix/patch! :)18:09
lbeliveaujaypipes: will retest everything and look at the race conditions, will most likely have results at summit18:10
jaypipeslbeliveau: excellent work.18:10
*** artom has quit IRC18:11
*** artom has joined #openstack-nova18:11
*** whenry has quit IRC18:12
*** artom has quit IRC18:12
*** artom has joined #openstack-nova18:12
*** hemanthm is now known as hemanthm|lunch18:13
openstackgerritLudovic Beliveau proposed openstack/nova: Fix drop_move_claim() on revert resize  https://review.openstack.org/34906018:14
*** tblakes has joined #openstack-nova18:14
lbeliveaujaypipes: ^ fixed commit message18:15
*** yamahata has quit IRC18:15
openstackgerritDan Smith proposed openstack/nova: Fix exception due to BDM race in get_available_resource()  https://review.openstack.org/34516218:16
openstackgerritAndreas Jaeger proposed openstack/nova: Enable release notes translation  https://review.openstack.org/38318918:17
*** bnemec has quit IRC18:17
*** tbachman has quit IRC18:17
jaypipeslbeliveau: thx. re-added my +2.18:19
*** slaweq_ has joined #openstack-nova18:20
*** mkoderer has quit IRC18:22
paroramriedem, can you please take a look at https://review.openstack.org/#/c/294513/ , it had a +2 but had some nits18:22
*** danpawlik has quit IRC18:23
*** roeyc has quit IRC18:23
*** sdague has joined #openstack-nova18:25
mriedemparora: yeah i'll try18:26
*** mat128|lunch is now known as mat12818:27
*** bnemec has joined #openstack-nova18:27
*** kfarr has quit IRC18:30
*** bnemec has quit IRC18:33
*** bnemec has joined #openstack-nova18:34
*** Sukhdev has joined #openstack-nova18:36
*** slaweq_ has quit IRC18:36
*** TravT has joined #openstack-nova18:37
*** ijw has joined #openstack-nova18:37
*** bnemec has quit IRC18:41
openstackgerritAndreas Jaeger proposed openstack/os-vif: Enable release notes translation  https://review.openstack.org/38332518:43
*** haplo37_ has quit IRC18:45
*** ijw has quit IRC18:45
*** ociuhandu has quit IRC18:46
*** haplo37_ has joined #openstack-nova18:47
*** krtaylor has joined #openstack-nova18:48
*** harlowja has quit IRC18:50
*** Pavo has quit IRC18:51
openstackgerritAndreas Jaeger proposed openstack/python-novaclient: Enable release notes translation  https://review.openstack.org/38338118:51
*** hwoarang has quit IRC18:51
*** hwoarang has joined #openstack-nova18:52
*** yamahata has joined #openstack-nova18:53
*** bnemec has joined #openstack-nova18:54
*** tbachman has joined #openstack-nova18:55
sdaguemriedem: https://review.openstack.org/#/c/351608/18 + W on that lands 4 conf cleanups18:55
*** Pavo has joined #openstack-nova18:56
mriedemlooking at this thing that dan cleaned up18:56
*** david-lyle has quit IRC18:57
*** kswiatek has quit IRC18:57
*** hwoarang has quit IRC18:57
*** hwoarang has joined #openstack-nova18:58
*** kswiatek has joined #openstack-nova18:58
*** raunak has quit IRC18:59
mriedemdansmith: the test in https://review.openstack.org/#/c/345162/ needs work18:59
*** TravT_ has joined #openstack-nova19:01
*** slaweq_ has joined #openstack-nova19:01
paroramriedem, thanks19:01
*** parora has quit IRC19:01
*** TravT has quit IRC19:02
*** TravT_ has quit IRC19:02
*** TravT has joined #openstack-nova19:02
*** jamielennox has quit IRC19:03
*** yamahata has quit IRC19:05
*** jamielennox|away has joined #openstack-nova19:05
*** jamielennox|away is now known as jamielennox19:06
*** ociuhandu has joined #openstack-nova19:06
*** Matti_ has joined #openstack-nova19:07
*** tonytan4ever has quit IRC19:07
*** yamahata has joined #openstack-nova19:08
*** ducttape_ has quit IRC19:09
*** mgoddard has joined #openstack-nova19:10
*** mgoddard_ has quit IRC19:10
*** bnemec has quit IRC19:10
*** rtheis_ has joined #openstack-nova19:11
*** bnemec has joined #openstack-nova19:11
*** artom has quit IRC19:11
*** cmartin has quit IRC19:11
*** rtheis has quit IRC19:12
*** salv-orlando has quit IRC19:13
slaweq_melwitt: hello19:13
*** artom has joined #openstack-nova19:13
slaweq_melwitt: some time ago we was talking about patch https://review.openstack.org/#/c/371592/2 (https://bugs.launchpad.net/nova/+bug/1623809), do You remember maybe?19:13
openstackLaunchpad bug 1623809 in OpenStack Compute (nova) "Quota exceeded when spawning instances in server group" [Undecided,In progress] - Assigned to Slawek Kaplonski (slaweq)19:13
*** annegentle has joined #openstack-nova19:13
slaweq_can You check in meantime my answear in this bug and check this patch? thx in advance19:14
*** mgoddard has quit IRC19:15
openstackgerritAndrew Laski proposed openstack/nova-specs: Conductor deployment options in cellsv2  https://review.openstack.org/38341919:16
*** hemanthm|lunch is now known as hemanthm19:16
*** coolsvap has quit IRC19:16
*** agireud has quit IRC19:21
*** READ10 has quit IRC19:23
*** tbachman has quit IRC19:24
mriedemmtreinish: sdague: we should get this into stable/newton as it impacts grenade jobs on master https://review.openstack.org/#/c/380645/19:25
*** hwoarang has quit IRC19:25
sdaguemriedem: looking...19:25
*** hamzy has quit IRC19:26
sdaguemriedem: lgtm19:26
mtreinishmriedem: +219:26
*** cmartin has joined #openstack-nova19:26
*** annegentle has quit IRC19:28
*** rama_y has quit IRC19:28
*** agireud has joined #openstack-nova19:29
*** sileht has quit IRC19:31
*** sileht has joined #openstack-nova19:32
*** hwoarang has joined #openstack-nova19:32
*** agireud has quit IRC19:33
*** penick has quit IRC19:34
*** jichen has joined #openstack-nova19:36
*** cmartin has quit IRC19:37
*** hamzy has joined #openstack-nova19:37
*** ijw has joined #openstack-nova19:37
*** ducttape_ has joined #openstack-nova19:39
jichenmriedem: hi, do you know if we want to create a non-voting 3rd party CI for nova drivers ,what kind of process or request should we make ?19:40
*** lpetrut has quit IRC19:40
jichenmriedem: we already have something like http://extbasicopstackcilog01.w3-969.ibm.com/test_logs/  , we want to make it non-voting job, but don't know where to start19:40
*** bvanhav has quit IRC19:40
mriedemjichen: i think here http://docs.openstack.org/infra/system-config/third_party.html19:42
jichenmriedem: yes, we consulted with CI folks and they said they focused on CI, and for nova specific task, we need ask inside nova19:43
*** agireud has joined #openstack-nova19:43
jichenso if we follow that page, there is nothing we need to do against nova itself?19:44
mriedemjichen: i don't believe so19:45
mriedemyou can comment on changes, just make sure they don't vote19:45
*** tonytan4ever has joined #openstack-nova19:45
mriedemhttps://review.openstack.org/#/admin/groups/511,members is the list of voting jobs19:46
jichenok, I see, maybe need some request to add us into nova-ci group and non-voting , I will see whether that info included in http://docs.openstack.org/infra/system-config/third_party.html or not19:47
*** penick has joined #openstack-nova19:47
*** dave-mccowan has quit IRC19:49
mriedemi don't believe your account needs to be in the nova-ci group19:49
mriedemthere are several 3rd party accounts that run jobs on nova changes which aren't in that group, like quobyte, virtuozzo, mellanox, etc19:49
*** Sukhdev has quit IRC19:51
jichenok, I see, I will go back check that again , thanks a lot for that info19:51
*** suresh12 has quit IRC19:51
*** ijw has quit IRC19:52
anteayajichen: yes, just read the information on that page about your gerrit account19:52
jichenanteaya: ok, will read that again, appreciate your info~19:52
anteayajichen: if you don't want to vote on patches just comment then you don't need to follow any process other than what is on that page19:52
*** zz_dimtruck is now known as dimtruck19:53
*** penick has quit IRC19:53
anteayajichen: if you have questions about that page you can find me in the #openstack-infra channel and I'll do the best I can to help you19:53
jichenanteaya: ok, thanks for offerings and will contact you when we have any issue :) , thanks19:54
*** penick has joined #openstack-nova19:54
anteayawelcome, and good luck19:54
mriedemsdague: +W on johns thing19:55
*** rama_y has joined #openstack-nova19:56
openstackgerritSlawek Kaplonski proposed openstack/nova: Fix quota calculation for instances in server_group  https://review.openstack.org/37159219:57
mriedemi've also gone through the list of stable/newton backports now that it's open https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/newton19:58
*** dave-mccowan has joined #openstack-nova19:58
*** dave-mcc_ has joined #openstack-nova20:01
*** haplo37 has quit IRC20:02
*** dimtruck is now known as zz_dimtruck20:02
*** dave-mccowan has quit IRC20:04
*** agireud has quit IRC20:06
*** david-lyle has joined #openstack-nova20:06
*** sdake has quit IRC20:09
*** haplo37 has joined #openstack-nova20:11
melwittmriedem: nice find on the hw_watchdog_action thing20:14
*** tonytan4ever has quit IRC20:16
*** harlowja has joined #openstack-nova20:17
mriedem\o/20:17
mriedemall sorts of terrible20:17
*** devananda|afk is now known as devananda20:18
*** code-R has joined #openstack-nova20:22
*** ihrachys has joined #openstack-nova20:23
openstackgerritDan Prince proposed openstack/nova: Fix API _workers options to be backwards compat  https://review.openstack.org/38344020:24
*** annegentle has joined #openstack-nova20:24
*** zz_dimtruck is now known as dimtruck20:24
*** anush_ has quit IRC20:25
*** salv-orlando has joined #openstack-nova20:27
openstackgerritKen'ichi Ohmichi proposed openstack/nova: Add error handling for delete-volume API  https://review.openstack.org/38266020:29
*** jichen has quit IRC20:30
*** cdent has quit IRC20:31
*** hemanthm is now known as hemanthm|bbl20:31
*** code-R_ has joined #openstack-nova20:31
*** suresh12 has joined #openstack-nova20:32
*** dimtruck is now known as zz_dimtruck20:34
*** code-R has quit IRC20:34
*** rtheis_ has quit IRC20:38
*** slaweq_ has quit IRC20:38
*** browne has joined #openstack-nova20:40
mriedemoomichi: so deleting a volume via the compute API proxy is deprecated20:41
*** dave-mcc_ has quit IRC20:41
mriedemoomichi: i'm not sure how much we should care to fix that20:41
oomichimriedem: yeah, I know that. I was a little concern about that.20:41
mriedemof course you could do it it with microversion<2.3620:41
oomichimriedem: but some tests of Tempest depends on the deprecated API now20:42
oomichimriedem: My original purpose is https://review.openstack.org/#/c/367358/20:43
oomichimriedem: that is not deprecated API20:43
oomichimriedem: when trying to reproduce the bug on the gate with https://review.openstack.org/#/c/382083/ , I found the proglem of https://review.openstack.org/#/c/382660/20:44
oomichis/proglem/problem/20:44
*** cmartin has joined #openstack-nova20:45
*** baoli_ has quit IRC20:46
mriedemoomichi: i thought tempest was getting rid of negative tests?20:48
mriedemi haven't been following that discussion really20:48
*** baoli has joined #openstack-nova20:48
oomichimriedem: that is a nice point. The conclusion is we can add some negative tests based on https://review.openstack.org/#/c/379013/20:49
oomichimriedem: I am thinking this case is integrated tests(between nova and cinder) and has value to avoid internal error. In addition, this is first negative test around this API20:50
*** Pavo has quit IRC20:51
oomichimriedem: thanks for nice review anyways :)20:52
oomichifor both nova and tempest20:52
mriedemoomichi: just posted anohther comment20:52
mriedemsure :)20:52
mriedemsince i had to add those create/attach volume common methods in tempest i have to -1 people for not using them now20:52
*** lucasagomes is now known as lucas-afk20:53
mriedemoomichi: so on https://review.openstack.org/#/c/382660/ i'd be ok with it if you didn't introduce the new exception type20:53
mriedemjust handle InvalidInput in the delete-volume REST API and we're good - it's much smaller20:53
oomichihehe, I know. maybe I also would get -1 on current one even if you don't point it20:53
mriedemthis 25 LOC patch is just way too big for me :)20:53
*** suresh12 has quit IRC20:55
*** Pavo has joined #openstack-nova20:55
*** FL1SK has quit IRC20:55
oomichimriedem: heh, yeah 25 LoC patch is huge ;)20:57
*** suresh12 has joined #openstack-nova20:57
*** TravT has quit IRC20:58
bauzasmriedem: about https://review.openstack.org/#/c/380068/20:59
bauzasmriedem : I think we can merge it now20:59
*** code-R_ has quit IRC20:59
bauzasSo the operators will see it20:59
mriedembauzas: seems kind of odd since it will show up in the 14.0.1 release21:00
*** whenry has joined #openstack-nova21:00
mriedembut it mentions 14.0.021:00
mriedemi'm not totally opposed to doing it now though21:00
bauzasMmm, I'll see21:01
*** dave-mccowan has joined #openstack-nova21:02
bauzasYou're right, it will be probably mentioning 14.0.-something21:02
*** pilgrimstack1 has joined #openstack-nova21:02
*** ayoung has quit IRC21:02
bauzasBut I'll see how it can be related to 14.021:02
openstackgerritMatt Riedemann proposed openstack/nova: Catch DevicePathInUse in attach_volume  https://review.openstack.org/36735821:03
mriedembauzas: i'm not sure anyone would care21:03
mriedemand it's actually more correct to say 14.0.0 even though the note wouldn't show up until 14.0.121:03
*** pilgrimstack has quit IRC21:04
*** Apoorva_ has joined #openstack-nova21:04
*** ijw has joined #openstack-nova21:04
mriedembauzas: ok +221:04
*** penick has quit IRC21:05
*** code-R has joined #openstack-nova21:07
*** penick has joined #openstack-nova21:07
*** Apoorva has quit IRC21:07
*** burgerk has quit IRC21:08
*** eharney has quit IRC21:09
*** ijw has quit IRC21:10
openstackgerritKen'ichi Ohmichi proposed openstack/nova: Add error handling for delete-volume API  https://review.openstack.org/38266021:10
*** annegentle has quit IRC21:13
*** baoli_ has joined #openstack-nova21:13
*** maticue has quit IRC21:14
*** tonytan4ever has joined #openstack-nova21:16
*** baoli has quit IRC21:17
*** harlowja has quit IRC21:17
*** rfolco has quit IRC21:18
*** tongli has quit IRC21:20
*** suresh12 has quit IRC21:20
openstackgerritMatt Riedemann proposed openstack/nova: Add error handling for delete-volume API  https://review.openstack.org/38266021:20
*** annegentle has joined #openstack-nova21:20
*** tonytan4ever has quit IRC21:22
*** baoli_ has quit IRC21:23
openstackgerritMatt Riedemann proposed openstack/nova: Delete traces of in-progress snapshot on VM being deleted  https://review.openstack.org/29451321:25
*** thorst_ has quit IRC21:27
*** thorst_ has joined #openstack-nova21:27
*** gouthamr has quit IRC21:27
*** thorst__ has joined #openstack-nova21:29
*** bdeschenes has quit IRC21:29
openstackgerritCharles Volzka proposed openstack/nova-specs: Add host-level memory monitor for scheduling  https://review.openstack.org/28881921:30
*** annegentle has quit IRC21:31
*** gouthamr has joined #openstack-nova21:31
*** code-R has quit IRC21:31
*** thorst_ has quit IRC21:32
*** gouthamr has quit IRC21:32
*** thorst__ has quit IRC21:34
*** david-lyle_ has joined #openstack-nova21:34
*** david-lyle has quit IRC21:34
*** david-lyle has joined #openstack-nova21:35
*** gouthamr has joined #openstack-nova21:35
*** david-lyle has quit IRC21:35
*** david-lyle has joined #openstack-nova21:35
*** annegentle has joined #openstack-nova21:36
*** gouthamr has quit IRC21:37
*** agireud has joined #openstack-nova21:37
*** esberglu has quit IRC21:37
*** mriedem has quit IRC21:38
*** pradk has quit IRC21:39
*** harlowja has joined #openstack-nova21:39
*** harlowja has quit IRC21:39
*** david-lyle_ has quit IRC21:39
*** harlowja has joined #openstack-nova21:39
openstackgerritDan Smith proposed openstack/nova: Fix exception due to BDM race in get_available_resource()  https://review.openstack.org/34516221:39
*** moshele has joined #openstack-nova21:41
dansmithmriedem_meeting: so the reason I didn't do that ^ in the first place is that I was assuming we'd never get to the case where we had a running domain from libvirt, but no BDMs due to a save race from the API21:42
dansmithmriedem_meeting: which .. I can't really imagine happening, unless it's ramdisk-only domains or something21:42
*** krtaylor has quit IRC21:42
dansmithmriedem_meeting: but you're right, that's clearly where we are in the bug report21:42
dansmithmriedem_meeting: i.e., we can't have defined the domain in libvirt without having seen the BDMs associated with it21:43
dansmithmriedem_meeting: or maybe old instances in the db that have no BDMs?21:45
dansmiththe bug report is pretty useless21:46
*** rcernin has joined #openstack-nova21:47
*** bnemec has quit IRC21:47
*** jheroux has quit IRC21:49
*** TravT has joined #openstack-nova21:50
*** tblakes has quit IRC21:51
*** marst has quit IRC21:54
*** rfolco has joined #openstack-nova21:54
*** TravT has quit IRC21:55
*** bdeschenes has joined #openstack-nova21:55
*** cmartin has quit IRC21:56
*** david-lyle has quit IRC21:56
*** ducttape_ has quit IRC21:58
*** burt has quit IRC21:59
*** penick has quit IRC21:59
*** rcernin has quit IRC21:59
*** rcernin has joined #openstack-nova21:59
openstackgerritMichael Glaser proposed openstack/nova: Transform instance.unpause notifications  https://review.openstack.org/38066821:59
*** mdrabe has quit IRC22:04
*** penick has joined #openstack-nova22:04
*** ijw has joined #openstack-nova22:06
jlkSo...22:06
jlkI wonder if I"m seeing a bug here22:06
jlkwell22:07
*** akshai has quit IRC22:07
jlkI've made a snapshot of an instance. The instance flavor was an 80 gig flavor. The snapshot itself is 1.2G, but it has specs of min_disks 80.22:08
jlkWhen nova pulls it down to boot an instance from it, the file in instances/_base/ gets exploded to 80.22:09
cfriesen_is it actually 80, or is it a sparse file?22:09
jlkit may be a sparse file. What's a good way to test that?22:09
jlkeither way, it's counting against the free space on the system22:09
cfriesen_the file in instances/_base is a raw file, intended to be used as backing store by qcow222:10
cfriesen_stat <file>22:10
jlk  Size: 85899345920     Blocks: 2544288    IO Block: 4096   regular file22:11
*** moshele has quit IRC22:11
*** ijw has quit IRC22:11
cfriesen_it's sparse.  "blocks" is the number of disk blocks (512 bytes) actually used by the file.22:11
*** Apoorva_ has quit IRC22:12
*** sdake has joined #openstack-nova22:13
*** Apoorva has joined #openstack-nova22:13
*** TravT has joined #openstack-nova22:14
*** jamesdenton has quit IRC22:14
*** bdeschenes has quit IRC22:15
*** jaypipes has quit IRC22:16
jlkcfriesen_: so something in Nova is sparsing out the file? downloads, sparses it out, then uses it as a backing file?22:16
*** suresh12 has joined #openstack-nova22:17
*** ociuhandu has quit IRC22:17
*** sdague has quit IRC22:18
*** ayoung has joined #openstack-nova22:21
*** FL1SK has joined #openstack-nova22:21
cfriesen_jlk: I think nova just calls out to qemu-img to convert it to raw for use as the backing file22:24
*** zz_dimtruck is now known as dimtruck22:25
openstackgerritAugustina Ragwitz proposed openstack/nova: Add microversion support to api unit tests  https://review.openstack.org/38081022:25
openstackgerritAugustina Ragwitz proposed openstack/nova: Restrict Metadata to Lower Case Keys  https://review.openstack.org/32969722:25
cfriesen_jlk:  _create_images_and_backing() in nova/virt/libvirt/driver.py22:28
jlkyeah, so even stranger22:29
jlkthere is calls in nova to fallocate, if preallocate is turned on, and it makes a call to a disk file to fallocate, such as fallocate -n -l 171798691840 /opt/stack/data/nova/instances/1e2ee91c-8cf3-48d4-9dc5-87920b327936/disk22:29
jlkso that allocates blocks, but the size remains small22:30
jlkalmost the opposite of the base image22:30
jlkokay, and disk usage show it that way too22:30
*** ntpttr has joined #openstack-nova22:31
*** krtaylor has joined #openstack-nova22:31
*** vishwanathj has quit IRC22:32
jlkso I wonder which nova uses to sort out sizes22:32
cfriesen_I think that preallocate defaults to turned off though22:33
*** suresh12 has quit IRC22:33
cfriesen_if you preallocate I don't think the size will stay small22:34
*** suresh12 has joined #openstack-nova22:34
jlkWe preallocate22:34
jlkbecause we had some customers that would quickly run themselves out of disk22:34
cfriesen_from the fallocate() man page..."22:34
cfriesen_The22:34
cfriesen_       file size (as reported by stat(2)) will be changed if offset+len is22:34
cfriesen_       greater than the file size.22:34
jlkso the scenario I'm working with22:35
jlkcustomer made a snapshot of a 300 gig flavor instance. So the min_disk of the snapshot is 300g, even though it might be small actual size22:36
cfriesen_oh, wait...that "-n" option tells it to not change the length of the file.  that's weird22:36
jlkthen they booted a 300g flavor instance from that snapshot22:36
jlkso the instance disk gets fallocated up to 300G, and the image disk gets sparsed up to 300G22:36
openstackgerritAugustina Ragwitz proposed openstack/nova: Restrict Metadata to Lower Case Keys  https://review.openstack.org/32969722:37
cfriesen_jlk: ouch22:38
*** suresh12 has quit IRC22:38
cfriesen_jlk: rather than using preallocate we fixed up the "disk_available_least" calculations22:39
jlkto make things worse, I think one of the instances has done something inside the VM so that the copy on write has copied over a TON of the data. So it's really actually sized up to 264 gig22:39
mtreinishauggy: ^^^ is that passing now?22:39
jlkcfriesen_: oh? that's good to know!22:39
cfriesen_jlk: I don't think they're all upstream yet.  it's slow going getting resource tracking fixes pushed.22:40
jlkah22:40
*** TonyXu has quit IRC22:41
*** TonyXu has joined #openstack-nova22:42
auggymtreinish: sooooon22:43
auggymtreinish: i just got everything rebased and (i think?) working after the rebase and then i had to update for some deprecations22:43
auggymtreinish: once that works i'm moving over to your nova manage patch to get that rebased and updated22:44
jlkdammmit22:44
jlkI set the image min_disk to 5G, and I'm booting a 160G flavor, and yet the downloaded base image file is still getting sparsed up to 80G22:45
auggyjlk: https://review.openstack.org/gitweb?p=openstack/nova.git;a=commitdiff;h=bd982bf2d5f7529f2c61140a11fdf2ac8620971e ??22:48
auggynot sure if that applies22:48
auggy-                sent_meta['min_disk'] = min_disk22:49
auggy+                sent_meta['min_disk'] = max(root_gb or 0, min_disk)22:49
auggyoh wait that's in a test22:49
auggyjlk: nm i think i misread this patch22:49
*** ducttape_ has joined #openstack-nova22:49
*** annegentle has quit IRC22:49
mtreinishauggy: heh, I can handle the rebase just tell me when it's a good idea :)22:50
auggymtreinish: sure, i'll let you know when this works22:50
auggymtreinish: also there was some discussion in the spec about adding a warning when squashing data if there are conflicts22:50
*** Pavo has quit IRC22:50
auggyi haven't re-reviewed the nova manage stuff since we worked on this a million years ago haha22:51
mtreinishauggy: I don't think there is a unique constraint on any of the columns the manage command touches22:51
mtreinishbut it's been a while22:51
auggyaggregate metadata might have one??22:51
*** Pavo has joined #openstack-nova22:51
auggybut it's more about people just being aware that they have multiple keys with the same name22:51
auggyoh right, i recently put a suggestion to add unit test coverage just to see what happens in that case and figure out what we think should happen22:52
mtreinishauggy: but if that's something we need to add, it's easy enough to do22:52
*** ajo_ has quit IRC22:53
auggymtreinish: yeah it might be... i'm just pondering the potential impacts22:53
auggyi guess it's just a warn, not a show stopper22:54
*** ajo has joined #openstack-nova22:58
openstackgerritAugustina Ragwitz proposed openstack/nova: Add microversion support to api unit tests  https://review.openstack.org/38081023:00
*** salv-orlando has quit IRC23:01
*** gouthamr has joined #openstack-nova23:01
*** ihrachys has quit IRC23:03
*** mlavalle has quit IRC23:05
*** ayoung has quit IRC23:05
*** claudiub has quit IRC23:07
*** ijw has joined #openstack-nova23:07
*** nic has quit IRC23:09
*** portdirect has joined #openstack-nova23:12
*** ijw has quit IRC23:14
*** xyang1 has quit IRC23:18
openstackgerritmelanie witt proposed openstack/nova: Add quota related tables to the api database.  https://review.openstack.org/33181023:23
*** esberglu has joined #openstack-nova23:24
*** amotoki has quit IRC23:24
mtreinishauggy: hmm, not unique for agg meta either: http://paste.openstack.org/show/584750/23:25
mtreinishwell per aggregate it is23:26
mtreinishbut not generally23:26
auggykk i remember a unique contraint violation being thrown as part of the original bug23:26
*** priteau has quit IRC23:26
*** mriedem has joined #openstack-nova23:26
*** dimtruck is now known as zz_dimtruck23:27
*** zz_dimtruck is now known as dimtruck23:27
auggyhttps://bugs.launchpad.net/nova/+bug/153801123:28
openstackLaunchpad bug 1538011 in OpenStack Compute (nova) "Metadata keys are not case sensitive" [Medium,In progress] - Assigned to Augustina Ragwitz (auggy)23:28
auggyohhh right23:28
auggyUNIQUE KEY `uniq_aggregate_metadata0aggregate_id0key0deleted` (`aggregate_id`,`key`,`deleted`),23:28
mtreinishoh, and flavor extra specs has the same thing. Unique per flavor: http://paste.openstack.org/show/584751/23:28
mtreinishright, if we violate that sqlalchemy will throw an exception when it tries to do the update23:28
auggyyeah23:29
mtreinishI'm not sure that's something we want to downgrade to a warning though23:29
auggyso would that case be covered by adding a test case to your nova-manage command that attempts to squash Foo and FOO?23:29
auggyyeah..23:29
auggyyeah i guess let's just get the status quo behavior working, make sure we don't overwrite anything without the user being aware of it and then just get review feedback on it...23:30
auggywe'll need to document this anyways, so we nned to make sure we have a fixing the keys workflow that we know works23:32
mtreinishwell I'm not sure how we would want to handle a conflict like that, we really wouldn't want to take an opinion in that case23:33
mtreinishthat's something that'll always require manual intervention23:33
*** esberglu has quit IRC23:35
*** rcernin has quit IRC23:36
*** dimtruck is now known as zz_dimtruck23:36
auggyyeah i agree23:41
jlkI think I just need to figure out how to make a snapshot of a VM so that when booting from that snapshot nova doesn't sparse fill the base image file23:42
*** ducttape_ has quit IRC23:44
*** ducttape_ has joined #openstack-nova23:46
*** ayoung has joined #openstack-nova23:49
*** Swami has quit IRC23:50
*** esberglu has joined #openstack-nova23:51
*** esberglu has quit IRC23:51
*** thorst_ has joined #openstack-nova23:52
*** rfolco has quit IRC23:53
*** pcaruana has quit IRC23:55
*** thorst_ has quit IRC23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!