| *** zigo has quit IRC | 00:05 | |
| *** Bhujay has joined #openstack-sahara | 02:29 | |
| *** dave-mccowan has quit IRC | 02:29 | |
| *** Bhujay has quit IRC | 02:30 | |
| *** Bhujay has joined #openstack-sahara | 02:30 | |
| *** dave-mccowan has joined #openstack-sahara | 02:49 | |
| *** Bhujay has quit IRC | 03:41 | |
| *** rcernin has quit IRC | 04:24 | |
| *** Bhujay has joined #openstack-sahara | 04:37 | |
| *** rcernin has joined #openstack-sahara | 04:38 | |
| *** rcernin_ has joined #openstack-sahara | 05:17 | |
| *** rcernin has quit IRC | 05:19 | |
| *** pcaruana has joined #openstack-sahara | 05:41 | |
| *** Bhujay has quit IRC | 06:00 | |
| *** Bhujay has joined #openstack-sahara | 06:14 | |
| *** Bhujay has quit IRC | 06:15 | |
| *** Bhujay has joined #openstack-sahara | 06:16 | |
| *** Bhujay has quit IRC | 06:32 | |
| *** Bhujay has joined #openstack-sahara | 06:33 | |
| *** rcernin_ has quit IRC | 07:05 | |
| *** tosky has joined #openstack-sahara | 07:54 | |
| openstackgerrit | Luigi Toscano proposed openstack/sahara master: Force the format of ssh key to PEM, at least for now https://review.openstack.org/605028 | 10:00 |
|---|---|---|
| *** Bhujay has quit IRC | 10:31 | |
| *** Bhujay has joined #openstack-sahara | 10:32 | |
| *** rcernin_ has joined #openstack-sahara | 10:43 | |
| *** rcernin_ has quit IRC | 10:52 | |
| *** pcaruana has quit IRC | 11:15 | |
| *** Bhujay has quit IRC | 12:31 | |
| *** Bhujay has joined #openstack-sahara | 12:32 | |
| *** Bhujay has quit IRC | 12:33 | |
| *** Bhujay has joined #openstack-sahara | 12:33 | |
| openstackgerrit | Luigi Toscano proposed openstack/sahara master: Force the format of ssh key to PEM, at least for now https://review.openstack.org/605028 | 12:37 |
| openstackgerrit | bhujay kumar proposed openstack/sahara master: adds unit test for ssh_remote.replace_remote_line https://review.openstack.org/605085 | 14:13 |
| tosky | \o/ | 14:22 |
| Bhujay | hope the approach is right | 14:26 |
| *** Bhujay has quit IRC | 15:03 | |
| *** dave-mccowan has quit IRC | 15:14 | |
| *** dave-mccowan has joined #openstack-sahara | 15:38 | |
| openstackgerrit | weizj proposed openstack/openstack-ansible-os_sahara master: Update messaging notification configuration https://review.openstack.org/605111 | 15:40 |
| *** wlmbasson has quit IRC | 15:52 | |
| openstackgerrit | weizj proposed openstack/openstack-ansible-os_sahara master: Update messaging notification configuration https://review.openstack.org/605111 | 15:58 |
| *** pcaruana has joined #openstack-sahara | 16:07 | |
| *** dave-mccowan has quit IRC | 16:13 | |
| tosky | tellesnobrega (and jeremyfreudberg, and any other core :) https://review.openstack.org/#/c/605028/ fixes an issue with openssl 1.1, which blocks the newer distributions (like Fedora 28) | 16:27 |
| tosky | I think I explained why I went for that solution instead of trying to use the new format (tl;dr paramiko does not support the new format) | 16:27 |
| tosky | but please try to keep an eye on it | 16:27 |
| tellesnobrega | tosky, saw it earlier, was waiting on tests | 16:29 |
| tosky | oki :) | 16:31 |
| tosky | I wrote it down so the other lurking people know about it | 16:31 |
| tellesnobrega | makes sense | 16:46 |
| tellesnobrega | looks good | 16:46 |
| tellesnobrega | tosky, are you still around? | 18:22 |
| *** guimaluf has joined #openstack-sahara | 18:59 | |
| guimaluf | hi folks, I'm using sahara on pike version and I just can't delete any cluster. Try to find something on web but could not find nothing. Any path to follow? | 19:00 |
| tellesnobrega | guimaluf, we have a bug about cluster deletion | 19:01 |
| tellesnobrega | this is due to some bad communication with other services | 19:01 |
| tellesnobrega | common approach now is to force delete heat stack | 19:01 |
| guimaluf | tellesnobrega, anything I can do about it or contribute to fix? | 19:02 |
| tellesnobrega | and than trying to delete to delete the cluster again | 19:02 |
| guimaluf | but there is no heat stack created | 19:02 |
| guimaluf | all clusters triggered Error on creation | 19:02 |
| tellesnobrega | guimaluf, we have fixes for newer versions | 19:02 |
| tellesnobrega | not sure we backported to pike | 19:02 |
| tellesnobrega | I see | 19:02 |
| guimaluf | tellesnobrega, thank you for the info! :) | 19:03 |
| tellesnobrega | on that case, since there is no stack created is the worst case | 19:03 |
| guimaluf | tellesnobrega, I've tried to remove direct on DB, but there is a foreignkey restriction | 19:04 |
| tellesnobrega | you would have to delete it directly on the DB | 19:04 |
| tellesnobrega | you can remove that reference as well | 19:04 |
| tellesnobrega | it won't break anything | 19:04 |
| guimaluf | thank you :) | 19:04 |
| guimaluf | there is no deleted field on clusters table... so I should delete from clusters; right? | 19:05 |
| tellesnobrega | yes | 19:06 |
| tellesnobrega | and this will be reject due to foreign key | 19:06 |
| tellesnobrega | but it shows the issue and you can delete that as well | 19:06 |
| tellesnobrega | and than come back to delete from clusters | 19:06 |
| guimaluf | tellesnobrega, I'm not fluent on database management ehhehe the best practice here would be `drop from node_groups; drop from clusters;` ? | 19:08 |
| tellesnobrega | try deleting from clusters first | 19:08 |
| tellesnobrega | and print show me the error message | 19:08 |
| tosky | tellesnobrega: yep? | 19:08 |
| guimaluf | ERROR 1451 (23000): Cannot delete or update a parent row: a foreign key constraint fails (`sahara`.`node_groups`, CONSTRAINT `node_groups_ibfk_1` FOREIGN KEY (`cluster_id`) REFERENCES `clusters` (`id`)) | 19:09 |
| tellesnobrega | you should delete from node groups and than clusters | 19:10 |
| tellesnobrega | tosky, about the tests on sahara side | 19:10 |
| guimaluf | I have no working cluster :) so I'll cleanup everything | 19:10 |
| tellesnobrega | that works | 19:11 |
| tellesnobrega | I'm down to 5 failing tests | 19:12 |
| tellesnobrega | and those 5 are fixed with the same fix | 19:12 |
| guimaluf | tellesnobrega, thank you :) | 19:12 |
| tellesnobrega | but I'm not sure how to do it | 19:12 |
| tellesnobrega | guimaluf, no problem :) | 19:13 |
| tellesnobrega | anytime | 19:13 |
| guimaluf | it works... I hope next time should be more complicated, since I want a working cluster xD | 19:13 |
| tellesnobrega | guimaluf, what happened that the cluster didn't start? | 19:13 |
| guimaluf | I was getting a random KeyError | 19:14 |
| tosky | guimaluf: that exact KeyError and the error message would help a lot in finding out if the issue is still there | 19:15 |
| tosky | is it the last version of the stable/pike branch, or some specific tag? | 19:15 |
| guimaluf | tosky, I'm pasting on paste.openstack :) | 19:15 |
| guimaluf | tosky, stable/pike | 19:15 |
| guimaluf | http://paste.openstack.org/show/730742/ | 19:16 |
| guimaluf | I don't know if this is image related or if is my setup | 19:16 |
| tosky | probably configuration-related | 19:22 |
| tosky | are you sure you have the tip of stable/pike? | 19:22 |
| tosky | because that error reminds me of this fix: https://review.openstack.org/#/c/502101/ | 19:23 |
| tosky | which is not in 7.0.0 (the first release of pike), but it's in 7.0.1 | 19:23 |
| tosky | (and newer releases) | 19:23 |
| tellesnobrega | tosky, it does looks familiar to that | 19:24 |
| tosky | https://storyboard.openstack.org/#!/story/1715681 | 19:24 |
| guimaluf | tosky, 1:7.0.0-0ubuntu1 | 19:25 |
| guimaluf | sahara is already the newest version (1:7.0.0-0ubuntu1~cloud0). | 19:26 |
| tosky | guimaluf: that's definitely not the last version; please check if the repository that you used provides a new er version | 19:26 |
| guimaluf | tosky, I'm using ubuntu cloud archive | 19:26 |
| tosky | it's the newest *packaged* version in that repository | 19:26 |
| guimaluf | yeah, :/ | 19:26 |
| tosky | I suggest to ping the maintainer of that archive to update their packages | 19:26 |
| tosky | we do release newer versions for a reason... | 19:26 |
| guimaluf | "<tosky> we do release newer versions for a reason..." that's the point! hahahah | 19:27 |
| tosky | you can manually apply that patch to your system, of course, at least you should be able to proceed | 19:28 |
| tosky | or change the configuration to add default values to those configuration keys (so that you don't need to patch the code) | 19:28 |
| tellesnobrega | tosky, https://github.com/tellesnobrega/sahara/tree/split-plugins | 19:29 |
| tellesnobrega | if you have the time can you take a look at this and see the failing tests | 19:29 |
| guimaluf | tosky, thank you for the hint! I'll apply the patch! :) | 19:30 |
| Gaasmann | have you heard of an issue where sahara keeps nc processes? I had a controller with 500 processes like that | 19:59 |
| Gaasmann | sudo sahara-rootwrap /etc/sahara/rootwrap.conf ip netns exec qdhcp-7e5c352e-7e71-412c-bb44-b2b846fe8f95 nc 10.42.0.5 22 | 19:59 |
| Gaasmann | even if the related clusters and instances were deleted | 19:59 |
| * tosky points Gaasmann to tellesnobrega and tries to hide | 20:00 | |
| Gaasmann | :-) | 20:00 |
| tellesnobrega | Gaasmann, looks like you have netns configured for sahara | 20:01 |
| Gaasmann | I think so yes | 20:01 |
| tellesnobrega | I'm not sure why they are kept, but those processes created for each instance or proxy instance that sahara needs to communicate | 20:02 |
| tellesnobrega | Gaasmann, we can take a look to try and make a better management of these processes | 20:04 |
| Gaasmann | makes sense as sahara connects to instances with ssh. But I don't know why they remain. I had exactly 500 of them, like there is a maximum or something | 20:05 |
| tellesnobrega | could be system | 20:06 |
| tellesnobrega | I will have to check | 20:06 |
| Gaasmann | ok thanks | 20:07 |
| tellesnobrega | no problem, thanks for bringing it up | 20:07 |
| Gaasmann | btw, for ambari, should I try to build centos or centos7 images? I can't get a running cluster with centos7 :-( | 20:08 |
| tellesnobrega | queens? | 20:08 |
| Gaasmann | pike :-/ | 20:08 |
| tellesnobrega | let me check | 20:09 |
| tellesnobrega | centos7 would be the best | 20:09 |
| tellesnobrega | what issue are you seeing? | 20:09 |
| Gaasmann | for for HDP 2.5, it's the exact same thing than I had with Ubuntu | 20:10 |
| Gaasmann | for the others, I don't know, the cluster is being configured and then a timeout occured but I can't get the action that timed out | 20:11 |
| tellesnobrega | :( | 20:11 |
| tellesnobrega | The timeout for ambari is a common issue, we have a new patch that allows increasing the timeout | 20:12 |
| tosky | centos 6 images were deprecated (and probably not even working) even in pike (centos 7 was the way already) | 20:12 |
| tellesnobrega | https://review.openstack.org/#/c/593598/ | 20:13 |
| tosky | tellesnobrega: should we start backporting it? | 20:14 |
| tellesnobrega | tosky, yes | 20:14 |
| tosky | tellesnobrega: technically it was marked as "feature" in the release notes, but it's more a fix for slow systems | 20:16 |
| tellesnobrega | it is a fix, we are basically exposing a config option | 20:17 |
| tellesnobrega | we can backport it | 20:17 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara stable/rocky: Add template param for ambari pkg install timeout https://review.openstack.org/605226 | 20:18 |
| guimaluf | should I use images within here http://sahara-files.mirantis.com/images/upstream/ to use sahara? or there is a better image repo? | 20:30 |
| guimaluf | I would like to get a sahara-spark with spark 2.x | 20:31 |
| tellesnobrega | guimaluf, the best practice is for you to generate your images | 20:32 |
| tellesnobrega | but I think for pike the images there would work (I believe) | 20:32 |
| guimaluf | tellesnobrega, should I use sahara elements or the newer image-gen? | 20:32 |
| tellesnobrega | guimaluf, newer image gen is for queens forward, for pike you should use sahara-image-elements | 20:33 |
| guimaluf | tellesnobrega, ok! :) nice :) thanks again xD | 20:33 |
| tosky | guimaluf: we removed all references to those images from our documentation, so no, please create your own images | 20:34 |
| guimaluf | tosky, no problem... there is a safe setup for creating my images? I did generate an spark image but I'm getting an error when creating the cluster | 20:35 |
| guimaluf | Error during command execution: "mv temp-file-aced7642-e5af-4887-a0a8-600342b396d7 /opt/hadoop/etc/hadoop/core-site.xml" | 20:35 |
| guimaluf | maybe I forgot something | 20:35 |
| tosky | make sure to use the last version of sahara-image-elements for the pike branch | 20:36 |
| tosky | and run it on a separate, fresh system (a small VM/instance is more then enough) | 20:36 |
| *** pcaruana has quit IRC | 20:43 | |
| tosky | tellesnobrega: those exceptions: I suspect that the "results" of the fake plugin for validation do not match the results expected by the test | 20:58 |
| tellesnobrega | tosky, that is what I think as well | 20:59 |
| tellesnobrega | not really sure how to properly fix, because some of those tests are interesting but not with fake plugin | 20:59 |
| tosky | couldn't the fake plugin be changed to simulate some more complex validation rules? | 21:03 |
| *** rcernin has joined #openstack-sahara | 23:07 | |
| *** tosky has quit IRC | 23:11 | |
| *** dave-mccowan has joined #openstack-sahara | 23:19 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!