| *** rcernin has quit IRC | 05:30 | |
| *** rcernin has joined #openstack-sahara | 05:30 | |
| *** rcernin_ has joined #openstack-sahara | 06:04 | |
| *** rcernin has quit IRC | 06:06 | |
| *** rcernin has joined #openstack-sahara | 06:34 | |
| *** rcernin_ has quit IRC | 06:36 | |
| *** rcernin has quit IRC | 07:06 | |
| *** hoonetorg has quit IRC | 07:17 | |
| *** hoonetorg has joined #openstack-sahara | 07:30 | |
| *** tosky has joined #openstack-sahara | 07:37 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara master: Deploy Sahara with unversioned endpoints https://review.openstack.org/583310 | 08:33 |
|---|---|---|
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_sahara stable/rocky: Deploy Sahara with unversioned endpoints https://review.openstack.org/603642 | 11:02 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_sahara stable/rocky: Move Sahara test playbook into role https://review.openstack.org/602654 | 11:03 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_sahara stable/rocky: Remove galera_client meta-dependency https://review.openstack.org/603644 | 11:03 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_sahara stable/rocky: Remove galera_client meta-dependency https://review.openstack.org/603644 | 11:03 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_sahara stable/queens: Remove galera_client meta-dependency https://review.openstack.org/603646 | 11:05 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_sahara stable/pike: Remove galera_client meta-dependency https://review.openstack.org/603647 | 11:05 |
| tellesnobrega | tosky, thanks | 11:34 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_sahara stable/rocky: Deploy Sahara with unversioned endpoints https://review.openstack.org/603642 | 14:19 |
| *** pcaruana has joined #openstack-sahara | 15:47 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara stable/pike: Remove galera_client meta-dependency https://review.openstack.org/603647 | 16:27 |
| *** pcaruana has quit IRC | 16:27 | |
| Gaasmann | I have a problem with clusters stuck in deleting mode. One of them has stacks in DELETE_FAILED status. But there is no server associated with any of those clusters. Is there a way to get rid of the clusters? | 18:09 |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara stable/queens: Remove galera_client meta-dependency https://review.openstack.org/603646 | 18:27 |
| tellesnobrega | Gaasmann, this is an issue that we have been trying to fix but it is not entirely our fault | 18:29 |
| tellesnobrega | can you try to force delete the stack? | 18:29 |
| *** vkmc is now known as vkmc|afk | 18:49 | |
| Gaasmann | tellesnobrega: I deleted the failed stack but I'm still seeing the clusters in Deleting mode on dataprocessing cluster list | 18:50 |
| tellesnobrega | hmm | 18:52 |
| tellesnobrega | I don't remember the correct approach here to fix this, but did you try after force deleting the stack did you try deleting the cluster again? | 18:54 |
| Gaasmann | no, let me try that | 18:54 |
| tellesnobrega | sure | 18:55 |
| Gaasmann | tellesnobrega: one of the cluster is gone. I can see in the logs that sahara-api complained about no finding the stack | 18:59 |
| tellesnobrega | that is expected | 18:59 |
| Gaasmann | For the other one, I didn't see any stack related | 18:59 |
| tellesnobrega | both were on the same stack? | 18:59 |
| Gaasmann | I don't think so | 19:00 |
| Gaasmann | when asking to delete the cluster, I see this message: | 19:00 |
| Gaasmann | 2018-09-19 18:57:13.827 19 DEBUG sahara.service.ops [req-ba3758a9-250e-4af1-9d61-633d13f01039 c2d70c68ef50422f84add4a7b107f8ae 43850a29dfee4b70b02edbfbc8d7c1eb - - -] [instance: none, cluster: 0df7ac04-b862-4ddf-b4f6-3ab805707999] Cluster was deleted or marked for deletion. Canceling current operation. wrapper /var/lib/kolla/venv/local/lib/python2.7/site-packages/sahara/service/ops.py:199 | 19:00 |
| Gaasmann | Not sure is current operation is the one running on the cluster during my asking for deletion or my deletion order | 19:01 |
| Gaasmann | s/is/if/ | 19:01 |
| tellesnobrega | not sure, probably the deletion operation | 19:02 |
| Gaasmann | I don't see any stack for the remaining cluster, even on list "stack list --hidden --nested --deleted" | 19:03 |
| tellesnobrega | that is weird, if there is no stack it should be deleted | 19:05 |
| Gaasmann | The cluster never worked. It went in error during validation because a process was missing | 19:07 |
| Gaasmann | sahara-api.log:2018-09-19 15:59:34.348 33 ERROR sahara.utils.api [req-cfeb6217-d4dd-4249-b3d4-dc5d2e8a2997 c2d70c68ef50422f84add4a7b107f8ae 43850a29dfee4b70b02edbfbc8d7c1eb - - -] [instance: none, cluster: 0df7ac04-b862-4ddf-b4f6-3ab805707999] Validation Error occurred: error_code=400, error_message=Cluster is missing a service: historyserver, required by service: oozie | 19:07 |
| Gaasmann | So I don't think it has even spawned a stack | 19:07 |
| tellesnobrega | I see | 19:08 |
| tellesnobrega | it should be deleted for sure | 19:08 |
| tellesnobrega | I think we fixed this validation on newer versions (it will be checked during template creation) | 19:09 |
| tellesnobrega | so the cluster won't fail to start on it | 19:09 |
| Gaasmann | For now, I don't think this cluster has any ressource (stack, vm or anything else) attached to it. So I can't see what prevents it from disappearing | 19:12 |
| Gaasmann | Is there a way to force it to go away? Because it prevents me from modifying the node group templates | 19:13 |
| tellesnobrega | I have a feature to force delete clusters, but I don't think is available on your deployment | 19:15 |
| Gaasmann | I'm on stable/pike :-/ | 19:15 |
| tellesnobrega | if you really need to force delete, I would suggest going into database and deleting the cluster | 19:16 |
| tosky | I guess you already tried to delete it again and to restart sahara-engine | 19:17 |
| Gaasmann | ok, is there any other resource than stack and VMs that a cluster can create? I'd like to check everything is clean before deleting on DB | 19:18 |
| Gaasmann | tosky: yep I tried both | 19:18 |
| Gaasmann | maybe I could try to change the Status back to Error and retry? | 19:19 |
| tellesnobrega | could work | 19:22 |
| tellesnobrega | not really sure | 19:22 |
| Gaasmann | it didn't work so I deleted the row and the related ones on node_groups | 19:43 |
| tellesnobrega | now is it gone? | 19:47 |
| Gaasmann | yes it is | 20:17 |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara stable/rocky: Move Sahara test playbook into role https://review.openstack.org/602654 | 21:02 |
| *** hoonetorg has quit IRC | 22:41 | |
| *** rcernin has joined #openstack-sahara | 22:44 | |
| *** hoonetorg has joined #openstack-sahara | 22:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!