| *** karthikpr has quit IRC | 00:41 | |
| *** karthikpr has joined #openstack-watcher | 01:01 | |
| *** karthikpr has quit IRC | 01:02 | |
| *** zhurong has joined #openstack-watcher | 01:11 | |
| *** sanfern has quit IRC | 01:49 | |
| *** karthikpr has joined #openstack-watcher | 01:53 | |
| *** karthikpr has quit IRC | 02:42 | |
| *** karthikpr has joined #openstack-watcher | 02:42 | |
| *** sanfern has joined #openstack-watcher | 03:03 | |
| *** sanfern has quit IRC | 03:11 | |
| *** sanfern has joined #openstack-watcher | 03:14 | |
| *** ianychoi has joined #openstack-watcher | 03:46 | |
| *** karthikpr has quit IRC | 04:23 | |
| *** karthikpr has joined #openstack-watcher | 04:27 | |
| *** karthikpr has quit IRC | 04:45 | |
| openstackgerrit | Santhosh Fernandes proposed openstack/watcher master: Add gnocchi support in basic_consolidation strategy https://review.openstack.org/440293 | 04:57 |
|---|---|---|
| *** karthikpr has joined #openstack-watcher | 05:03 | |
| *** thorst_afk has joined #openstack-watcher | 05:19 | |
| *** thorst_afk has quit IRC | 05:19 | |
| *** thorst_afk has joined #openstack-watcher | 05:28 | |
| *** sanfern has quit IRC | 05:32 | |
| *** thorst_afk has quit IRC | 05:32 | |
| *** sanfern has joined #openstack-watcher | 05:32 | |
| *** thorst_afk has joined #openstack-watcher | 05:43 | |
| *** thorst_afk has quit IRC | 05:47 | |
| *** sanfern has quit IRC | 06:36 | |
| *** sanfern has joined #openstack-watcher | 06:37 | |
| *** dtardivel has joined #openstack-watcher | 06:53 | |
| *** aspiers has quit IRC | 08:08 | |
| *** aspiers has joined #openstack-watcher | 08:11 | |
| *** openstackgerrit has quit IRC | 08:18 | |
| dtardivel | Yumeng: Hi Yumeng | 08:26 |
| Yumeng | dtardivel:hi David | 08:26 |
| *** vincentfrancoise has joined #openstack-watcher | 08:26 | |
| dtardivel | Yumeng: I reviewed your last version of spec last friday. | 08:27 |
| Yumeng | dtardivel: Yes, I have noticed that! Many thanks | 08:28 |
| dtardivel | Part of comments was just about trying to understand your use case from a technical point of view. | 08:29 |
| Yumeng | and I was just discussing with licanwei | 08:29 |
| dtardivel | If you want to use Ironic, it's ok. But we have to do it properly, by building a baremetal data model | 08:30 |
| dtardivel | and parameters for Action 'power off/on' should be extract from this model. | 08:32 |
| *** licanwei has joined #openstack-watcher | 08:37 | |
| Yumeng | dtardivel: ironic is not used for baremetal management in this spec. if you mind discussing later? I will add a diagram into the spec to show why we want to use ironic which might be easier to be understood. | 08:53 |
| *** alexchadin has joined #openstack-watcher | 10:08 | |
| *** licanwei has quit IRC | 10:13 | |
| *** openstackgerrit has joined #openstack-watcher | 10:21 | |
| openstackgerrit | Merged openstack/watcher-specs master: Add team priorites for Pike https://review.openstack.org/442472 | 10:21 |
| *** zhurong has quit IRC | 10:21 | |
| *** zhurong has joined #openstack-watcher | 10:21 | |
| *** sanfern has quit IRC | 10:34 | |
| *** dtardivel has quit IRC | 11:30 | |
| *** alexchadin has quit IRC | 11:46 | |
| *** alexchadin has joined #openstack-watcher | 11:47 | |
| *** figleaf is now known as edleafe | 12:21 | |
| openstackgerrit | zhongshengping proposed openstack/puppet-watcher master: Fix typo https://review.openstack.org/444872 | 12:23 |
| *** zhurong has quit IRC | 13:01 | |
| *** ragahv has joined #openstack-watcher | 13:25 | |
| ragahv | hi | 13:25 |
| ragahv | i am facing problem while running audit | 13:26 |
| ragahv | when i run audit i get error The cluster state is not defined | 13:26 |
| ragahv | can anyone help to resolve this issue..or tell me whether i missed some configuration | 13:26 |
| vincentfrancoise | ragahv: hi | 13:27 |
| ragahv | Hi vincent | 13:28 |
| ragahv | req-34955670-b106-4615-ae69-7211b5dedd6d None None] The cluster state is not defined 2017-03-13 14:52:49.160 TRACE watcher.decision_engine.audit.base Traceback (most recent call last): 2017-03-13 14:52:49.160 TRACE watcher.decision_engine.audit.base File "/opt/stack/watcher/watcher/decision_engine/audit/base.py", line 130, in execute 2017-03-13 14:52:49.160 TRACE watcher.decision_engine.audit.base solution = self.do_e | 13:28 |
| vincentfrancoise | ragahv: can you give some detail/logs regarding your issue | 13:28 |
| vincentfrancoise | ? | 13:28 |
| ragahv | yes | 13:28 |
| ragahv | 2017-03-13 14:52:49.160 ERROR watcher.decision_engine.audit.base [req-34955670-b106-4615-ae69-7211b5dedd6d None None] The cluster state is not defined 2017-03-13 14:52:49.160 TRACE watcher.decision_engine.audit.base Traceback (most recent call last): 2017-03-13 14:52:49.160 TRACE watcher.decision_engine.audit.base File "/opt/stack/watcher/watcher/decision_engine/audit/base.py", line 130, in execute 2017-03-13 14:52:49.160 | 13:28 |
| ragahv | when i run audit | 13:29 |
| ragahv | i get this | 13:29 |
| vincentfrancoise | can you also give me the version/commit hash of your Watcher? | 13:29 |
| vincentfrancoise | ragahv: ^ | 13:29 |
| ragahv | i am using ocata release | 13:30 |
| ragahv | is that you are asking | 13:30 |
| vincentfrancoise | ragahv: ok thanks | 13:32 |
| vincentfrancoise | ragahv: I already have my guess, just verifying ;) | 13:32 |
| ragahv | vincent: ok | 13:33 |
| vincentfrancoise | ragahv: ok so my guess is that there is a bug that was found during the last few days before the final release which was backported to the ocata branch that you probably do not have | 13:36 |
| vincentfrancoise | ragahv: watcher 1.0.0 has this bug whereas v1.0.1 has it fixed | 13:37 |
| openstackgerrit | zhongshengping proposed openstack/puppet-watcher master: Fix ci https://review.openstack.org/444905 | 13:37 |
| vincentfrancoise | ragahv: https://bugs.launchpad.net/watcher/+bug/1662810 | 13:38 |
| openstack | Launchpad bug 1662810 in watcher "model_root.get_node_instances(node) returns none instances" [Critical,Fix released] - Assigned to licanwei (li-canwei2) | 13:38 |
| vincentfrancoise | ragahv: can you check if you have this fix on your side? | 13:39 |
| openstackgerrit | zhongshengping proposed openstack/puppet-watcher master: Fix ci https://review.openstack.org/444905 | 13:42 |
| ragahv | Vincent: Fix is in / watcher/decision_engine/model/collector/nova.py file | 13:44 |
| ragahv | right | 13:44 |
| ragahv | vincent:i have watcher version 1.0.1 | 13:46 |
| ragahv | vincent:and i could see fix present | 13:46 |
| vincentfrancoise | ragahv: ok good | 13:46 |
| ragahv | vincent: but still i am getting error | 13:46 |
| vincentfrancoise | ragahv: do you get this error everytime? or is it pseudo-random? | 13:47 |
| ragahv | vincent: everytime | 13:48 |
| vincentfrancoise | ragahv: can you send me the Watcher logs for the decision-engine process you have when the error occurs? | 13:49 |
| ragahv | i have shared it above | 13:49 |
| openstackgerrit | Merged openstack/watcher-specs master: Define when an action plan is stale/invalid https://review.openstack.org/432193 | 13:50 |
| vincentfrancoise | ragahv: the error you get is a consequence of another error that should have occured in the background | 13:51 |
| vincentfrancoise | ragahv: so the error you get should have logged another error before crashing during the execution of your audit | 13:51 |
| vincentfrancoise | ragahv: and this is this initial error I am asking you to find if you can ;) | 13:51 |
| ragahv | vincent: ok..i will rerun it and check | 13:52 |
| vincentfrancoise | ragahv: thanks | 13:52 |
| ragahv | vincent:please find below log | 13:55 |
| vincentfrancoise | can you paste them on http://paste.openstack.org/ please? | 13:55 |
| vincentfrancoise | vfrancoise: will be easier to read for me ;) | 13:55 |
| ragahv | vincent:pasted..i could one notification error | 13:57 |
| vincentfrancoise | ragahv: can't see anything | 13:57 |
| ragahv | vincent: in logs before the error i reported | 13:57 |
| ragahv | raghav: i have pasted log | 13:58 |
| ragahv | vincent: i have pasted the log | 13:58 |
| vincentfrancoise | ragahv: need to send the resulting link ;) | 13:58 |
| ragahv | vincent: http://paste.openstack.org/show/602497/ | 13:58 |
| vincentfrancoise | ragahv: thx | 13:58 |
| vincentfrancoise | ragahv: did you configure the nova notifications as described in https://docs.openstack.org/developer/watcher/deploy/configuration.html#configure-nova-notifications ? | 14:00 |
| ragahv | vincent:yes its same | 14:02 |
| *** karthikpr has joined #openstack-watcher | 14:07 | |
| vincentfrancoise | alexchadin: ^ any guess? | 14:09 |
| alexchadin | hi ragahv | 14:10 |
| ragahv | hi alex | 14:10 |
| *** karthikpr has quit IRC | 14:10 | |
| alexchadin | ragahv: did you take these logs from decision engine logs? | 14:14 |
| ragahv | alex: yes | 14:14 |
| vincentfrancoise | ragahv: BTW, when I re-read the logs you sent, the error I was referring to should be another one | 14:15 |
| alexchadin | ragahv: isn't there any other logs from API? | 14:15 |
| vincentfrancoise | ragahv: notifications.error is just Watcher notifying whoever is interested that things didn't go well | 14:15 |
| ragahv | vincent: yes | 14:16 |
| vincentfrancoise | ragahv: in case you want to plugin it to some supervision tool or 3rd party software | 14:16 |
| alexchadin | vincentfrancoise: maybe background update of CDM isn't good? | 14:16 |
| vincentfrancoise | alexchadin: yes that's what I think | 14:16 |
| vincentfrancoise | alexchadin: but doesn't tell me what is going wrong | 14:17 |
| alexchadin | we need more infos | 14:17 |
| alexchadin | oh | 14:17 |
| vincentfrancoise | alexchadin: 1st option I see, the CDM is stale | 14:17 |
| vincentfrancoise | alexchadin: 2nd option, some option: a notification-related CDM update crashes | 14:18 |
| vincentfrancoise | alexchadin: 3rd option: The background job that rebuilds the CDM from scratch every hour or so is crashing | 14:19 |
| vincentfrancoise | alexchadin: I would personally think it's option 2 or 3 but we need more as you said | 14:19 |
| *** karthikpr has joined #openstack-watcher | 14:26 | |
| ragahv | Vincent: hi | 14:33 |
| ragahv | vincent: any clue | 14:33 |
| vincentfrancoise | ragahv: can you give us the full trace of what you get from the moment you restart your decision-engine process until you get the error given that you enabled the debug mode in your config file? | 14:38 |
| ragahv | vincent: ok sure | 14:38 |
| vincentfrancoise | ragahv: thx | 14:39 |
| alexchadin | ragahv: it would be great if you provide us logs from the very beginning of service working | 14:43 |
| ragahv | alex:sure | 14:44 |
| *** karthikpr has quit IRC | 14:50 | |
| ragahv | alex: log is very big | 14:51 |
| vincentfrancoise | ragahv: if it fits, send it all please | 14:51 |
| ragahv | vincent: is there any way i can share log file | 14:52 |
| vincentfrancoise | what's the overal size of the log? | 14:52 |
| ragahv | vincent:199k | 14:53 |
| ragahv | vincent: till the ppoint we reach error message it nearly 650 lines | 14:54 |
| ragahv | vincent: i have cut some code and pasted some portion of entire code | 14:56 |
| ragahv | vincent: http://paste.openstack.org/show/602508/ | 14:56 |
| alexchadin | vincentfrancoise: lol | 15:01 |
| alexchadin | vincentfrancoise: have you found it too? | 15:01 |
| alexchadin | vincentfrancoise: 105 line | 15:01 |
| *** karthikpr has joined #openstack-watcher | 15:10 | |
| *** karthikpr has quit IRC | 15:11 | |
| vincentfrancoise | alexchadin: yes 00mValueError: invalid literal for int() with base 10: 'region!child@1' | 15:21 |
| ragahv | vincent: is it issue related to some package | 15:23 |
| *** ianychoi has quit IRC | 15:29 | |
| vincentfrancoise | ragahv: don't know why yet, but it seems like you have a field in nova that Watcher couldn't parse "region!child@1" | 15:31 |
| vincentfrancoise | ragahv: problem is that I can't tell which one it is that contains this | 15:32 |
| ragahv | vincent: ok ..i wil try to ..If you find anything please let me know | 15:36 |
| *** alexchadin has quit IRC | 15:36 | |
| vincentfrancoise | ragahv: can you run a CLI command for me please | 15:37 |
| ragahv | vincent: yeh tell | 15:37 |
| vincentfrancoise | ragahv: and tell me where in nova you see this "region!child@1" value | 15:37 |
| *** karthikpr has joined #openstack-watcher | 15:39 | |
| *** ianychoi has joined #openstack-watcher | 15:40 | |
| vincentfrancoise | ragahv: should be displayed in "openstack hypervisor list --long" or in "openstack hypervisor show <UUID>" | 15:41 |
| ragahv | yes its the ID in hypervisor list | 15:42 |
| vincentfrancoise | ragahv: ok so I get it now... | 15:42 |
| vincentfrancoise | ragahv: that's a bug on our side | 15:43 |
| vincentfrancoise | ragahv: Thanks for telling us | 15:43 |
| ragahv | vincent: ohh | 15:43 |
| vincentfrancoise | ragahv: can you check if it works if I tell you the fix? | 15:43 |
| ragahv | vincent : sure | 15:44 |
| vincentfrancoise | ragahv: this line https://github.com/openstack/watcher/blob/master/watcher/decision_engine/model/element/node.py#L35 | 15:44 |
| vincentfrancoise | ragahv: should actually be '"id": wfields.StringField(),' | 15:45 |
| vincentfrancoise | ragahv: this way, it should be able to parse the hypervisor ID as a string and not an int | 15:46 |
| ragahv | vincent:ok i have updated in my code | 15:47 |
| ragahv | vincent: just restarting watcher decision engine is fine right? | 15:47 |
| vincentfrancoise | ragahv: yes | 15:47 |
| vincentfrancoise | ragahv: I am rechecking the nova API doc and it still seems odd to me though... | 15:48 |
| ragahv | vincent:Fixed | 15:49 |
| vincentfrancoise | ragahv: it worked? | 15:49 |
| ragahv | vincent: yes | 15:49 |
| vincentfrancoise | ragahv: ok now what I don't understand is why you have a string as hypervisor ID | 15:50 |
| vincentfrancoise | ragahv: referring to https://developer.openstack.org/api-ref/compute/?expanded=list-compute-services-detail,list-hosts-detail,list-hypervisors-detail#list-hypervisors | 15:51 |
| vincentfrancoise | ragahv: or even https://developer.openstack.org/api-ref/compute/?expanded=list-compute-services-detail,list-hosts-detail,list-hypervisors-detail,list-hypervisors-details-detail,show-hypervisor-details-detail#show-hypervisor-details | 15:51 |
| vincentfrancoise | ragahv: the ID field should be an integer | 15:51 |
| vincentfrancoise | ragahv: "id body integer The id of the hypervisor." | 15:52 |
| ragahv | vincent: yes...i too was going to ask you same..as its type is integer how come it came as string | 15:52 |
| vincentfrancoise | ragahv: so now I am not so sure the problem comes from Watcher anymore | 15:52 |
| vincentfrancoise | ragahv: what's the version of your Nova setup BTW? | 15:53 |
| ragahv | vincent:7.1.0 | 15:55 |
| vincentfrancoise | ragahv: isn't that that novaclient's version? | 15:56 |
| ragahv | vincent: yes | 15:56 |
| ragahv | vincent: its novaclient version | 15:57 |
| *** karthikpr has quit IRC | 15:58 | |
| *** karthikpr has joined #openstack-watcher | 16:00 | |
| ragahv | vincent:its 15.0.1 | 16:00 |
| vincentfrancoise | ragahv: ok so ocata version too then | 16:01 |
| ragahv | vincent: yes | 16:01 |
| vincentfrancoise | ragahv: well, this is odd... | 16:02 |
| vincentfrancoise | ragahv: could you ask the nova folks if they know why it's not an integer? | 16:02 |
| ragahv | vincent: whats problem | 16:02 |
| vincentfrancoise | ragahv: because the docs says integer so I cannot raise a bug if what you say goes against the Nova doc and I can't tell why... | 16:03 |
| ragahv | vincent:ok | 16:04 |
| vincentfrancoise | ragahv: also DB-wise, id is marked as an Integer in Nova codebase (see https://github.com/openstack/nova/blob/stable/mitaka/nova/objects/compute_node.py#L59) | 16:04 |
| vincentfrancoise | ragahv: ocata version https://github.com/openstack/nova/blob/stable/ocata/nova/objects/compute_node.py#L56 | 16:05 |
| ragahv | vincent: ok | 16:06 |
| *** karthikpr has quit IRC | 16:28 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-watcher master: Add bindep support https://review.openstack.org/444992 | 16:38 |
| *** karthikpr has joined #openstack-watcher | 16:50 | |
| *** vincentfrancoise has quit IRC | 17:14 | |
| *** openstackgerrit has quit IRC | 17:18 | |
| *** karthikpr has quit IRC | 17:37 | |
| *** thorst_afk has joined #openstack-watcher | 18:04 | |
| *** thorst_afk has quit IRC | 18:04 | |
| *** karthikpr has joined #openstack-watcher | 18:06 | |
| *** sanfern has joined #openstack-watcher | 18:09 | |
| *** karthikpr has quit IRC | 18:11 | |
| *** karthikpr has joined #openstack-watcher | 18:28 | |
| *** wootehfoot has joined #openstack-watcher | 19:43 | |
| *** openstackgerrit has joined #openstack-watcher | 19:44 | |
| *** karthikpr has quit IRC | 19:47 | |
| *** karthikpr has joined #openstack-watcher | 19:54 | |
| *** karthikpr has quit IRC | 20:16 | |
| *** wootehfoot has quit IRC | 20:24 | |
| *** wootehfoot has joined #openstack-watcher | 20:41 | |
| *** karthikpr has joined #openstack-watcher | 20:46 | |
| *** wootehfoot has quit IRC | 21:15 | |
| *** aspiers has quit IRC | 22:03 | |
| *** karthikpr has quit IRC | 22:12 | |
| *** karthikpr has joined #openstack-watcher | 22:17 | |
| *** sanfern has quit IRC | 22:30 | |
| *** sanfern has joined #openstack-watcher | 22:34 | |
| *** karthikpr has quit IRC | 22:36 | |
| *** karthikpr has joined #openstack-watcher | 22:37 | |
| *** wootehfoot has joined #openstack-watcher | 22:58 | |
| *** karthikpr has quit IRC | 22:59 | |
| *** karthikpr has joined #openstack-watcher | 22:59 | |
| *** karthikpr has quit IRC | 23:05 | |
| *** aspiers has joined #openstack-watcher | 23:37 | |
| *** wootehfoot has quit IRC | 23:37 | |
| *** karthikpr has joined #openstack-watcher | 23:54 | |
| *** karthikpr has quit IRC | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!