| SamYaple | yea thats the issue i remember. and if i recall it was too new container distro on your base distro | 00:01 |
|---|---|---|
| SamYaple | but like i said, it was centos only and i dont centos so couldnt reproduce | 00:01 |
| *** mnasiadka has joined #openstack-kolla | 00:08 | |
| *** bmace_ has quit IRC | 00:10 | |
| *** mnasiadka has quit IRC | 00:13 | |
| *** itlinux has joined #openstack-kolla | 00:16 | |
| *** mnasiadka has joined #openstack-kolla | 00:23 | |
| *** mnasiadka has quit IRC | 00:27 | |
| *** JamesBenson has joined #openstack-kolla | 00:28 | |
| *** caowei has joined #openstack-kolla | 00:32 | |
| *** JamesBenson has quit IRC | 00:33 | |
| *** caoyuan has joined #openstack-kolla | 00:37 | |
| *** mnasiadka has joined #openstack-kolla | 00:38 | |
| *** k_mouza has quit IRC | 00:41 | |
| *** k_mouza has joined #openstack-kolla | 00:41 | |
| *** mnasiadka has quit IRC | 00:42 | |
| *** mdnadeem has joined #openstack-kolla | 00:50 | |
| *** k_mouza has quit IRC | 00:51 | |
| *** mnasiadka has joined #openstack-kolla | 00:53 | |
| *** pengdake_ has joined #openstack-kolla | 00:55 | |
| *** zhaochao has joined #openstack-kolla | 00:55 | |
| *** mnasiadka has quit IRC | 00:58 | |
| *** sai___ has joined #openstack-kolla | 01:02 | |
| *** fungi has joined #openstack-kolla | 01:03 | |
| fungi | inc0: i've opened bug 1749326 as a semi-urgent request to stop exposing memcached to the internet in kolla-ansible-oraclelinux-* jobs | 01:05 |
| openstack | bug 1749326 in kolla-ansible "Exploitable services exposed on community test nodes" [Undecided,New] https://launchpad.net/bugs/1749326 | 01:05 |
| fungi | not sure who else in particular to give a heads-up on that one | 01:06 |
| inc0 | thanks | 01:06 |
| inc0 | I'll take a look | 01:06 |
| fungi | much appreciated! | 01:06 |
| *** mnasiadka has joined #openstack-kolla | 01:08 | |
| *** pengdake_ has quit IRC | 01:11 | |
| *** pengdake_ has joined #openstack-kolla | 01:11 | |
| *** mnasiadka has quit IRC | 01:13 | |
| *** awiddersheim has quit IRC | 01:13 | |
| *** awiddersheim has joined #openstack-kolla | 01:13 | |
| *** awiddersheim has joined #openstack-kolla | 01:13 | |
| *** pengdake has joined #openstack-kolla | 01:16 | |
| *** pengdake_ has quit IRC | 01:16 | |
| *** itlinux has quit IRC | 01:19 | |
| *** mnasiadka has joined #openstack-kolla | 01:23 | |
| *** hongbin has joined #openstack-kolla | 01:24 | |
| *** mnasiadka has quit IRC | 01:28 | |
| *** mnasiadka has joined #openstack-kolla | 01:38 | |
| *** mnasiadka has quit IRC | 01:43 | |
| *** itlinux has joined #openstack-kolla | 01:44 | |
| *** mnasiadka has joined #openstack-kolla | 01:53 | |
| *** dardelean has joined #openstack-kolla | 01:53 | |
| *** dardelean has quit IRC | 01:58 | |
| *** mnasiadka has quit IRC | 01:58 | |
| *** pengdake has quit IRC | 01:59 | |
| *** awiddersheim has quit IRC | 02:02 | |
| *** mnasiadka has joined #openstack-kolla | 02:08 | |
| *** zhaochao has quit IRC | 02:09 | |
| *** itlinux has quit IRC | 02:10 | |
| emccormick | Anyone notice the severity_label field has gone missing in kibana in Pike? | 02:12 |
| emccormick | Am I missing something? | 02:12 |
| emccormick | refreshed the field list, nuked indexes and refreshed again. Just seems not there. | 02:13 |
| *** mnasiadka has quit IRC | 02:13 | |
| *** signed8bit_Zzz is now known as signed8b_ | 02:17 | |
| *** zhaochao has joined #openstack-kolla | 02:22 | |
| *** mnasiadka has joined #openstack-kolla | 02:23 | |
| *** mnasiadka has quit IRC | 02:28 | |
| inc0 | fungi: still around? | 02:28 |
| *** itlinux has joined #openstack-kolla | 02:32 | |
| *** signed8b_ is now known as signed8bit_Zzz | 02:33 | |
| fungi | yeah | 02:35 |
| *** pengdake has joined #openstack-kolla | 02:36 | |
| *** awiddersheim has joined #openstack-kolla | 02:37 | |
| *** mnasiadka has joined #openstack-kolla | 02:39 | |
| *** mdnadeem has quit IRC | 02:42 | |
| fungi | inc0: did you have a question for m? | 02:42 |
| fungi | e? | 02:42 |
| *** mnasiadka has quit IRC | 02:43 | |
| *** itlinux has quit IRC | 02:44 | |
| *** salv-orl_ has joined #openstack-kolla | 02:45 | |
| *** salv-orlando has quit IRC | 02:48 | |
| *** mnasiadka has joined #openstack-kolla | 02:54 | |
| fungi | well, i have to get some sleep but will be around off and on tomorrow | 02:58 |
| *** chmarkus has quit IRC | 02:58 | |
| *** mnasiadka has quit IRC | 02:59 | |
| *** chmarkus has joined #openstack-kolla | 02:59 | |
| *** mnasiadka has joined #openstack-kolla | 03:09 | |
| *** unicell has quit IRC | 03:09 | |
| *** mnasiadka has quit IRC | 03:13 | |
| *** awiddersheim has quit IRC | 03:17 | |
| *** awiddersheim has joined #openstack-kolla | 03:17 | |
| *** k_mouza has joined #openstack-kolla | 03:19 | |
| *** mnasiadka has joined #openstack-kolla | 03:23 | |
| *** k_mouza has quit IRC | 03:24 | |
| *** gkadam has joined #openstack-kolla | 03:24 | |
| *** awiddersheim has quit IRC | 03:28 | |
| *** mnasiadka has quit IRC | 03:28 | |
| *** harlowja has quit IRC | 03:30 | |
| *** awiddersheim has joined #openstack-kolla | 03:30 | |
| *** devananda has quit IRC | 03:31 | |
| *** pengdake has quit IRC | 03:33 | |
| *** pengdake has joined #openstack-kolla | 03:33 | |
| *** mnasiadka has joined #openstack-kolla | 03:38 | |
| *** caowei has quit IRC | 03:39 | |
| *** awiddersheim has quit IRC | 03:41 | |
| *** jtriley has joined #openstack-kolla | 03:42 | |
| *** skramaja has joined #openstack-kolla | 03:42 | |
| *** mnasiadka has quit IRC | 03:43 | |
| *** mnasiadka has joined #openstack-kolla | 03:47 | |
| *** skramaja has quit IRC | 03:50 | |
| *** skramaja has joined #openstack-kolla | 03:50 | |
| *** mnasiadka has quit IRC | 03:52 | |
| *** mnasiadka has joined #openstack-kolla | 03:53 | |
| *** mnasiadka has quit IRC | 03:58 | |
| *** mnasiadka has joined #openstack-kolla | 04:08 | |
| *** pengdake has quit IRC | 04:08 | |
| *** goldyfruit has quit IRC | 04:09 | |
| *** dave-mccowan has quit IRC | 04:10 | |
| *** harlowja has joined #openstack-kolla | 04:12 | |
| *** gkadam_ has joined #openstack-kolla | 04:13 | |
| *** mnasiadka has quit IRC | 04:13 | |
| *** gkadam_ has quit IRC | 04:14 | |
| *** gkadam_ has joined #openstack-kolla | 04:14 | |
| *** gkadam has quit IRC | 04:15 | |
| *** mnasiadka has joined #openstack-kolla | 04:23 | |
| *** prernadembla05 has quit IRC | 04:27 | |
| *** mnasiadka has quit IRC | 04:28 | |
| *** harlowja has quit IRC | 04:30 | |
| *** mnasiadka has joined #openstack-kolla | 04:38 | |
| *** lpetrut has joined #openstack-kolla | 04:38 | |
| *** mnasiadka has quit IRC | 04:44 | |
| *** olivierbourdon38 has joined #openstack-kolla | 04:49 | |
| *** hongbin has quit IRC | 04:55 | |
| *** pengdake has joined #openstack-kolla | 05:01 | |
| *** pengdake has quit IRC | 05:01 | |
| *** pengdake has joined #openstack-kolla | 05:02 | |
| *** jtriley has quit IRC | 05:03 | |
| *** lpetrut has quit IRC | 05:11 | |
| *** harlowja has joined #openstack-kolla | 05:14 | |
| *** jtriley has joined #openstack-kolla | 05:14 | |
| *** caoyuan has quit IRC | 05:16 | |
| *** lpetrut has joined #openstack-kolla | 05:19 | |
| *** pengdake has quit IRC | 05:26 | |
| *** lpetrut has quit IRC | 05:29 | |
| *** janki has joined #openstack-kolla | 05:36 | |
| *** ktibi has quit IRC | 05:50 | |
| *** signed8bit_Zzz has quit IRC | 05:52 | |
| *** coolsvap has joined #openstack-kolla | 06:17 | |
| *** pengdake has joined #openstack-kolla | 06:18 | |
| *** signed8bit has joined #openstack-kolla | 06:22 | |
| *** masber has joined #openstack-kolla | 06:22 | |
| *** harlowja has quit IRC | 06:24 | |
| *** pengdake has quit IRC | 06:33 | |
| *** mnasiadka has joined #openstack-kolla | 06:43 | |
| *** unicell has joined #openstack-kolla | 06:57 | |
| *** dciabrin has joined #openstack-kolla | 06:57 | |
| *** unicell has quit IRC | 06:57 | |
| *** unicell has joined #openstack-kolla | 06:58 | |
| *** unicell1 has joined #openstack-kolla | 07:02 | |
| *** unicell has quit IRC | 07:03 | |
| *** egonzalez has joined #openstack-kolla | 07:04 | |
| *** jchhatbar has joined #openstack-kolla | 07:05 | |
| *** janki has quit IRC | 07:09 | |
| *** jchhatbar is now known as janki | 07:14 | |
| *** sai___ has quit IRC | 07:22 | |
| *** pengdake has joined #openstack-kolla | 07:23 | |
| *** hamza21 has joined #openstack-kolla | 07:29 | |
| *** mdnadeem has joined #openstack-kolla | 07:30 | |
| *** lvdombrkr has joined #openstack-kolla | 07:31 | |
| *** pengdake has quit IRC | 07:39 | |
| *** lpetrut has joined #openstack-kolla | 07:42 | |
| *** b_bezak has joined #openstack-kolla | 07:46 | |
| *** dardelean has joined #openstack-kolla | 07:52 | |
| *** pcaruana has joined #openstack-kolla | 07:56 | |
| *** paken has joined #openstack-kolla | 08:03 | |
| paken | Hi, good morning | 08:04 |
| *** masber has quit IRC | 08:05 | |
| paken | I submitted the following change request few days ago https://review.openstack.org/541700 | 08:05 |
| *** robbbe has quit IRC | 08:05 | |
| paken | Actually is my first one; we have others to submit, but I wanted to start with this one, for being the first change request to the group | 08:06 |
| paken | The change was set to -1, and I gave an explanation of the request afterward, but I didn’t get any reply anymore. Is there any reviewer here that I can talk to? Cheers! | 08:07 |
| *** dardelean has quit IRC | 08:07 | |
| *** emccormick has quit IRC | 08:07 | |
| *** dardelean has joined #openstack-kolla | 08:07 | |
| paken | We want to use Kolla-Ansible at our company, but we would like to discuss some changes to the playbooks. Thanks! | 08:07 |
| *** dardelean has quit IRC | 08:12 | |
| *** unicell1 has quit IRC | 08:13 | |
| *** gfidente has joined #openstack-kolla | 08:14 | |
| *** gfidente has quit IRC | 08:14 | |
| *** gfidente has joined #openstack-kolla | 08:14 | |
| *** lvdombrkr89 has joined #openstack-kolla | 08:23 | |
| *** k_mouza has joined #openstack-kolla | 08:24 | |
| *** lvdombrkr has quit IRC | 08:26 | |
| *** k_mouza has quit IRC | 08:28 | |
| *** mgoddard_ has joined #openstack-kolla | 08:43 | |
| *** gkadam__ has joined #openstack-kolla | 08:50 | |
| *** gkadam_ has quit IRC | 08:53 | |
| hrw | morning | 08:56 |
| hrw | paken: first of all: if you want people to review then write why it is needd in commit message | 08:57 |
| hrw | paken: my rule is "if I have to read code to know why change is sent then it may get to the bottom of queue" | 08:58 |
| paken | ok, great, thanks for the feedback | 08:59 |
| *** dardelean has joined #openstack-kolla | 08:59 | |
| *** dardelean has quit IRC | 09:00 | |
| *** dardelean has joined #openstack-kolla | 09:00 | |
| *** ktibi has joined #openstack-kolla | 09:01 | |
| hrw | paken: imho now it is time for you to read comments, understand them and reply with some improvements | 09:01 |
| hrw | paken: I have no idea how openstack<>ceph works so can not help here | 09:01 |
| paken | sure, will do. | 09:03 |
| paken | Actually I replied into the comments on the 8th of February | 09:04 |
| paken | I don’t know if this is the right way, or just submitting directly the new code with the improvement. | 09:05 |
| hrw | I usually go with both | 09:05 |
| hrw | reply to comments and then push new version for review | 09:05 |
| paken | ok, perfect, thanks! | 09:06 |
| hrw | yw | 09:07 |
| *** Angeh has joined #openstack-kolla | 09:11 | |
| *** jmccarthy has joined #openstack-kolla | 09:14 | |
| *** janki has quit IRC | 09:22 | |
| *** dardelean has quit IRC | 09:26 | |
| *** dardelean has joined #openstack-kolla | 09:28 | |
| *** owalsh has left #openstack-kolla | 09:33 | |
| *** k_mouza has joined #openstack-kolla | 09:34 | |
| *** dardelean has quit IRC | 09:40 | |
| yankcrime | i still need some love for https://review.openstack.org/#/c/534834/ | 09:43 |
| gema | paken: keep improving the patch with the feedback you received and add a commit explanation indeed | 09:43 |
| yankcrime | it's valentines day after all | 09:43 |
| gema | paken: the other thing you can do is socialize this change at the weekly team meeting | 09:43 |
| gema | when the floor opens for comments | 09:43 |
| gema | or add your item to the agenda and come and talk to the team about it | 09:43 |
| gema | paken: I am not sure I understand why you need that change | 09:43 |
| gema | paken: nor in which context you have tested it/validated it | 09:44 |
| *** k_mouza has quit IRC | 09:46 | |
| paken | gema: I wanted to avoid the hardcoding of the names of the keyrings in the tasks, and take the name to inventory. | 09:46 |
| paken | gema: In our company we have some Openstack clusters using the same Ceph cluster | 09:47 |
| gema | paken: that's what you are doing, but why are you doing it | 09:47 |
| gema | ah, ok | 09:47 |
| gema | paken: knowing your usecase is useful | 09:47 |
| paken | gema: Every cinder of every Openstack cluster uses a different pool | 09:47 |
| gema | paken: ack | 09:47 |
| paken | gema: and we want to take the name of the pool to the keyring name | 09:47 |
| gema | paken: the other thing to note is that kolla is going to move away from ceph at the EOL of luminous | 09:48 |
| gema | we will be moving to ceph-ansible | 09:48 |
| paken | I should have included this in the commit message | 09:48 |
| gema | paken: so you may want to look at that as well | 09:48 |
| *** k_mouza has joined #openstack-kolla | 09:48 | |
| gema | (you have a few releases still for that) | 09:48 |
| paken | do you have the github link so that I can have a look? | 09:48 |
| gema | paken: sure, google has it ;) https://github.com/ceph/ceph-ansible | 09:49 |
| paken | thanks ;-) | 09:50 |
| gema | paken: if you are migrating now, it may be worth using that directly, I am not sure | 09:50 |
| gema | paken: we are also going to be embracing kolla soon and that's something we are considering | 09:50 |
| paken | gema: we actually have already a Ceph cluster, we just want to configure the frontend part | 09:52 |
| paken | so, all the pools are already set | 09:52 |
| paken | and we want the keyrings to be something like ceph.client.<POOL_NAME>.keyring | 09:53 |
| paken | or <RBD_USERNAME> | 09:53 |
| paken | for administration/clarification purposes | 09:53 |
| gema | paken: sounds reasonable, upgrade from brownfield deployments is something we as a team are very interested in, we also have a ceph cluster we need to upgrade | 09:54 |
| gema | paken: it may be interested if you could blog about this migration | 09:54 |
| lvdombrkr89 | hello,quick question when i bash to containers in one container i bash as a root, but in other as a local user (mysql for example) | 09:54 |
| gema | paken: we are planning to do that | 09:54 |
| lvdombrkr89 | question is why there is difference? | 09:55 |
| lvdombrkr89 | why i cant bash all containers as a root? | 09:55 |
| *** aikema has joined #openstack-kolla | 09:58 | |
| paken | gema: sure, will do, let me first submit the improvement | 09:58 |
| gema | paken: sounds good! | 09:59 |
| paken | gema: afterward I will submit a bigger change about the standarization of the different external_ceph.yml, because now in nova, glance, cinder, manila, gnnochi, they are extremely different | 09:59 |
| paken | gema: nova/tasks/external_ceph.yml checks for the existence of the keyrings, but glance and cinder don’t | 10:00 |
| gema | paken: bear in mind people are using these scripts in production, so you need to consider backwards compatibility | 10:00 |
| gema | paken: and upgradability from previous versions | 10:00 |
| paken | gema: ok, I’ll take it in account, thanks! | 10:01 |
| gema | np | 10:01 |
| *** shardy has joined #openstack-kolla | 10:02 | |
| egonzalez | lvdombrkr89, it depends on the service, some of them need to run as root (nova_libvirt) others as the user (nova_api) | 10:05 |
| *** pcaruana has quit IRC | 10:05 | |
| egonzalez | lvdombrkr89, you can enter as root with docker exec -ti -u 0 <container> bash | 10:05 |
| egonzalez | -u (user) 0 (uuid 0 root) | 10:05 |
| lvdombrkr89 | egonzalez: thanks, yes i know how to switch , question was about why is difference.. but you answered anyway | 10:07 |
| egonzalez | lvdombrkr89, user is defined in the image https://github.com/openstack/kolla/blob/master/docker/blazar/blazar-api/Dockerfile.j2#L19 if no user is defined is root | 10:08 |
| egonzalez | mandre, around | 10:10 |
| aikema | still trying to debug why I've been unable to get mariadb working in an ubuntu-based deployment. I see in https://review.openstack.org/#/c/531656/ that kolla-ansible-ubuntu-source reports success but kolla-ansible-ubuntu-source-ceph lists as failure, but with broken links. Can anyone explain this and the difference between the two? | 10:12 |
| mandre | hey egonzalez, what's up? | 10:12 |
| egonzalez | mandre, remember the denver ptg topic for deployment tools about common healthchecks effort? know if there is any progress on the topic? | 10:14 |
| egonzalez | aikema, change was already merged, difference between ceph and non ceph jobs is that ceph runs a multinode deployment | 10:15 |
| mandre | egonzalez: yeah I remember, I even found the etherpad - https://etherpad.openstack.org/p/tripleo-ptg-queens-kolla-collaboration | 10:17 |
| *** caoyuan has joined #openstack-kolla | 10:17 | |
| mandre | egonzalez: I didn't follow up closely, to my knowledge there was no progress this cycle | 10:17 |
| aikema | egonzalez: would that suggest that an allinone deploy would be expected to work on Ubuntu but not a multinode deploy (at least as of time of that issue)? | 10:18 |
| egonzalez | aikema, multinode works | 10:18 |
| egonzalez | aikema, during that time zuul had many issues, restarting, disk space, etc. now are resolved afik | 10:19 |
| egonzalez | aikema, probably that job failure was a random error not related to the change | 10:19 |
| aikema | egonzalez: OK. Will clueless after about a couple of weeks of digging why my attempts to deploy a stable/pike kolla using ubuntu are still failing with mariadb issues | 10:20 |
| egonzalez | aikema, check this one, a change from yesterday all test passed https://review.openstack.org/#/c/543488/ | 10:20 |
| mandre | egonzalez: it would be nice to come up with a plan to upstream the healthchecks to kolla at this PTG | 10:20 |
| *** pcaruana has joined #openstack-kolla | 10:20 | |
| mandre | i won't be present unfortunately, but maybe larks or mmagr will | 10:21 |
| egonzalez | mandre, so bad :( | 10:22 |
| *** skramaja has quit IRC | 10:22 | |
| egonzalez | added a topic for cross-community goals work | 10:23 |
| egonzalez | aikema, where/when is failing? | 10:23 |
| lvdombrkr89 | egonzalez: thanks, its clear now | 10:23 |
| aikema | egonzalez: So if I find seemingly the latest stable/pike version (you'd linked to master - i.e. queens), I see https://review.openstack.org/#/c/544085/. It shows an "kolla-ansible-ubuntu-source" not an kolla-ansible-ubuntu-source-ceph. Is stable/pike also tested as multinode if that's what you were reporting was the diff b/w those labels in that test? | 10:24 |
| *** zhaochao has quit IRC | 10:24 | |
| egonzalez | aikema, looking for the job gates, we reworked them, probably missed ceph | 10:24 |
| egonzalez | aikema, i'll work backporting the changes to pike on the evening, now are in merge conflict | 10:28 |
| aikema | egonzalez: Thanks | 10:29 |
| *** dardelean has joined #openstack-kolla | 10:30 | |
| egonzalez | aikema, anyway, nothing im aware of changed in mariadb for pike | 10:31 |
| egonzalez | aikema, what are your errors? | 10:31 |
| aikema | egonzalez: basically this I think https://bugs.launchpad.net/kolla/+bug/1732314 | 10:34 |
| openstack | Launchpad bug 1732314 in kolla "mariadb Connection refused" [Undecided,New] | 10:34 |
| aikema | (bug originally reported by someone else back in November, but it seems to match the symptoms I'm seeing) | 10:35 |
| egonzalez | aikema, mind sharing your globals.yml, pip freeze and docker ps -a on controller hosts, and any mariadb container log (if there are any bootstrap_mariadb too) | 10:37 |
| *** serlex has joined #openstack-kolla | 10:42 | |
| aikema | egonzalez: globals.yml: https://pastebin.com/jAdJLpzt / pip freeze from a controller node: https://pastebin.com/h0fQbxQ3 | 10:43 |
| egonzalez | aikema, pip freeze in deploy host , sorry | 10:44 |
| aikema | I think I killed off the last deploy partway through the waiting-for-slave timeouts from the other mariadb nodes... but here's docker ps -a on the deploy control group: https://pastebin.com/1uq7vdXS | 10:46 |
| egonzalez | aikema, first thing i see in globals: same interface for network, external_vip and neutron_external | 10:46 |
| aikema | egonzalez: been trying various permutations - this was trimmed down from an earlier config which was using a broader range of network interfaces but that made no difference | 10:47 |
| aikema | egonzalez: pip freeze on deployment host: https://pastebin.com/38KUHUPS | 10:48 |
| *** sambetts|afk is now known as sambetts | 10:48 | |
| egonzalez | aikema, even if mariadb passes, when reaching openvswitch will lose connection to the hosts due the interface is going to be used by ovs br-ex | 10:49 |
| aikema | egonzalez: figured I'd worry about that if I can get mariadb started | 10:50 |
| egonzalez | aikema, check keepalived logs too | 10:50 |
| egonzalez | maybe is something wrong using same interface for internal and external vip with different IPs on same interface | 10:50 |
| *** k_mouza has quit IRC | 10:50 | |
| aikema | egonzalez: earlier was using different interfaces for this and different IP ranges as well for each | 10:52 |
| openstackgerrit | Martin Mágr proposed openstack/kolla master: Add socat to base image https://review.openstack.org/543624 | 10:58 |
| *** k_mouza has joined #openstack-kolla | 10:58 | |
| *** dasTor_ has joined #openstack-kolla | 11:14 | |
| openstackgerrit | Vladislav Belogrudov proposed openstack/kolla-ansible master: Don't check api_interface for 'deployment' group https://review.openstack.org/543959 | 11:17 |
| *** dasTor has quit IRC | 11:17 | |
| *** serlex has quit IRC | 11:29 | |
| *** pbourke has quit IRC | 11:29 | |
| *** pbourke has joined #openstack-kolla | 11:30 | |
| *** pcaruana has quit IRC | 11:30 | |
| *** pcaruana has joined #openstack-kolla | 11:42 | |
| pbourke | yankcrime: apologies for the inertia on your review | 11:43 |
| pbourke | yankcrime: I'm really keen to resolve the questions around config but I realise you're stuck in the middle | 11:44 |
| pbourke | yankcrime: if you can make it we have our team meeting today so it would be a good time to raise it again and see what the feeling is | 11:45 |
| openstackgerrit | Vladislav Belogrudov proposed openstack/kolla-ansible master: Don't check api_interface for 'deployment' group https://review.openstack.org/543959 | 11:53 |
| *** dave-mccowan has joined #openstack-kolla | 11:56 | |
| *** robbbe has joined #openstack-kolla | 12:07 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla-ansible master: Updated from global requirements https://review.openstack.org/544264 | 12:07 |
| *** gkadam has joined #openstack-kolla | 12:17 | |
| pbourke | are our images broken? | 12:17 |
| pbourke | seems the uuid provider has been removed in keystone queens | 12:17 |
| pbourke | I mean rocky | 12:18 |
| *** egonzalez has quit IRC | 12:18 | |
| *** gkadam__ has quit IRC | 12:19 | |
| yankcrime | pbourke: no worries, and i'll bring it up at the team meeting | 12:25 |
| pbourke | I added it to the agenda @ https://wiki.openstack.org/wiki/Meetings/Kolla | 12:26 |
| yankcrime | thanks | 12:26 |
| *** ktibi has quit IRC | 12:35 | |
| *** gkadam has quit IRC | 12:38 | |
| *** robbbe has quit IRC | 12:38 | |
| lvdombrkr89 | hello folks, one time i asked this question, maybe any ideas why my keystone db is up to 40gb and and old tokens not flushing automatically? | 12:46 |
| pbourke | lvdombrkr89: I believe you need to flush them manually | 12:46 |
| pbourke | lvdombrkr89: we should probably add a cron task to do this unsure if there's a reason we haven't already | 12:46 |
| lvdombrkr89 | pbourke: thanks, i need to remove all data directly from database? | 12:51 |
| pbourke | lvdombrkr89: no there's a command you can use | 12:51 |
| pbourke | lvdombrkr89: have a google and if you can't find it let me know | 12:51 |
| *** robbbe has joined #openstack-kolla | 12:52 | |
| pbourke | it seems this problem goes away with fernet | 12:52 |
| *** goldyfruit has joined #openstack-kolla | 12:55 | |
| *** pradk has joined #openstack-kolla | 12:56 | |
| lvdombrkr89 | pbourke: my env is on 4.0.0 tag last week i tried migrate from uuid to fernet to resolve this issue, but stuck on deployment and stay on uuid | 12:57 |
| pbourke | lvdombrkr89: we'll add a proper upgrade mechanism in rocky | 12:58 |
| pbourke | lvdombrkr89: what kind of issue(s) did you run into? | 12:58 |
| *** goldyfruit has quit IRC | 13:02 | |
| *** rhallisey has joined #openstack-kolla | 13:06 | |
| *** Angeh has quit IRC | 13:11 | |
| lvdombrkr89 | pbourke: i stuck in something like http://paste.openstack.org/raw/666349/ | 13:12 |
| pbourke | hmm that looks like keystone isn't up properly | 13:13 |
| pbourke | would need more info on how you tried to do the migration | 13:13 |
| lvdombrkr89 | pbourke: i just in globals.yaml swich keystone_token_provider from uuid to fernet and run reconfigure | 13:16 |
| pbourke | lvdombrkr89: ok - I would log a bug for this | 13:17 |
| lvdombrkr89 | pbourke: okey thanks, i can re-run migration to collect any logs if you need. | 13:20 |
| pbourke | lvdombrkr89: can you please link me the bug once you log it? | 13:21 |
| lvdombrkr89 | pbourke: sure, i will ping you later ) | 13:24 |
| pbourke | thanks! | 13:24 |
| lvdombrkr89 | pbourke: i tried to flush my tokens but get this : http://paste.openstack.org/raw/672161/ | 13:30 |
| *** pradk has quit IRC | 13:36 | |
| *** hrw has quit IRC | 13:45 | |
| lvdombrkr89 | pbourke: this is my keystone.conf: http://paste.openstack.org/raw/672177/ | 13:46 |
| *** hrw has joined #openstack-kolla | 13:46 | |
| *** Reepicheep has quit IRC | 13:46 | |
| *** Reepicheep has joined #openstack-kolla | 13:46 | |
| *** kiennt26 has joined #openstack-kolla | 13:49 | |
| *** hamza21 has quit IRC | 13:52 | |
| Reepicheep | I seem to be having an issue with my ceph setup. I set it up with some cacheing (writeback) drives using kolla-ansible but it seems it has not flushed any data to the any backend osd | 13:55 |
| Reepicheep | http://paste.openstack.org/show/672187/ | 13:55 |
| Reepicheep | Any idea how to get that to flush data? I'm not even sure where the ceph tiering agent is supposed to be running | 13:57 |
| *** goldyfruit has joined #openstack-kolla | 14:01 | |
| openstackgerrit | Merged openstack/kolla-ansible stable/pike: Increase vm.max_map_count on elasticsearch nodes https://review.openstack.org/544079 | 14:02 |
| *** jtriley has quit IRC | 14:02 | |
| *** caoyuan has quit IRC | 14:03 | |
| *** ktibi has joined #openstack-kolla | 14:07 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Support configuration of custom fluentd filters https://review.openstack.org/513445 | 14:13 |
| openstackgerrit | Merged openstack/kolla-ansible master: Fix the Prechecks failed while checking docker version https://review.openstack.org/533339 | 14:13 |
| *** egonzalez has joined #openstack-kolla | 14:20 | |
| paken | Could we add this to the today’s meeting? https://bugs.launchpad.net/kolla-ansible/+bug/1748194 | 14:20 |
| openstack | Launchpad bug 1748194 in kolla-ansible ""Running MariaDB bootstrap container" fails" [Undecided,New] | 14:20 |
| paken | It’s really weird that during the first deployment the mariadb container cannot start | 14:22 |
| paken | and after a second deployment it goes further and it’s the rabbitmq container that cannot start | 14:23 |
| paken | afterwards the same happens to keystone, glance… | 14:23 |
| *** mdnadeem has quit IRC | 14:24 | |
| *** rhallisey has quit IRC | 14:25 | |
| pbourke | paken: sure, feel free to edit the agenda wiki page | 14:26 |
| paken | pbourke: nice, will do | 14:27 |
| dtk | hello guys | 14:28 |
| dtk | @paken: what do you get if you run docker logs on that container ? | 14:29 |
| pbourke | hey dtk | 14:31 |
| paken | @dtk, I will run again from 0 the deploy and I will add it to the bug | 14:31 |
| paken | when it was run by kolla-ansible, then I saw the logs with ‘docker logs <MARIADB_CONTAINER>’ and I didn’t see any error | 14:32 |
| paken | afterward I ran manually “docker start -a bootstrap_mariadb” | 14:32 |
| paken | and I got “ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)” | 14:32 |
| dtk | is the database password set ? | 14:32 |
| dtk | docker logs bootstrap_mariadb | 14:32 |
| paken | I have the same situation as Jonathan in the bug | 14:33 |
| dtk | also, did you previously run kolla-genpwd ? | 14:33 |
| dtk | before the deployment ? | 14:33 |
| paken | I tried both ways | 14:34 |
| paken | using the passwords.yml of a previous installation | 14:34 |
| paken | and with a new one | 14:34 |
| dtk | can you also add the --verbose flag ? | 14:34 |
| paken | and then only editting the rbd related secret | 14:34 |
| dtk | somehow, I think MYSQL_ROOT_PASSWORD is not set | 14:36 |
| dtk | within the container | 14:36 |
| dtk | the bootstrap one | 14:36 |
| dtk | DB_ROOT_PASSWORD | 14:36 |
| dtk | this I mean | 14:36 |
| dtk | because it gives you access denied | 14:36 |
| dtk | can you add some lines inside /usr/local/share/kolla-ansible/roles/mariadb/tasls/bootstrap_cluster.yml ? | 14:37 |
| dtk | there's also a chance it just doesn't get read | 14:38 |
| paken | but that’s when I ran manually “docker start -a bootstrap_mariadb” | 14:38 |
| dtk | hmm, just do this then | 14:38 |
| paken | when the bootstrap was done by the playbooks, I got a non-zero exit (message from ansible) | 14:38 |
| paken | I checked the logs, and it finished correctly | 14:38 |
| dtk | after docker run set -e "DB_ROOT_PASSWORD={yourDBPassword}" | 14:39 |
| dtk | can you check what version of docker you have ? | 14:39 |
| *** coolsvap has quit IRC | 14:39 | |
| dtk | the module | 14:39 |
| dtk | I mean | 14:39 |
| dtk | pip list | grep docker | 14:39 |
| paken | docker (3.0.1) | 14:39 |
| paken | docker-pycreds (0.2.1) | 14:39 |
| dtk | that's it | 14:39 |
| dtk | :D | 14:39 |
| dtk | run this | 14:39 |
| paken | what version do I need? | 14:40 |
| dtk | ansible -i /path/to/inventory -m shell 'pip install docker==2.7.0 --upgrade' all | 14:40 |
| *** jtriley has joined #openstack-kolla | 14:40 | |
| dtk | from your deployment host | 14:40 |
| dtk | then try again | 14:40 |
| paken | ok, let me try | 14:40 |
| *** rhallisey has joined #openstack-kolla | 14:41 | |
| dtk | just to check, which version of kolla-ansible are you trying to deploy ? | 14:41 |
| dtk | pike by any chance ? | 14:41 |
| dtk | or master ? | 14:41 |
| paken | pike | 14:42 |
| paken | the point is that everything that worked last week, it doesn’t this one | 14:42 |
| dtk | yup, this should take care of it for you | 14:42 |
| paken | same globals.yml | 14:42 |
| dtk | docker-py released | 14:42 |
| paken | same passwords.yml | 14:42 |
| dtk | that's why it "surprised" you | 14:42 |
| dtk | docker-py and docker use the same namespace | 14:43 |
| paken | I’m running again the deployment | 14:44 |
| dtk | cool | 14:44 |
| dtk | lmk how it goes | 14:44 |
| dtk | but you should be good | 14:44 |
| openstackgerrit | zhongshengping proposed openstack/kolla-ansible stable/pike: Fix the Prechecks failed while checking docker version https://review.openstack.org/544363 | 14:45 |
| *** hamza21 has joined #openstack-kolla | 14:45 | |
| paken | dtk: yeap | 14:45 |
| openstackgerrit | zhongshengping proposed openstack/kolla-ansible stable/ocata: Fix the Prechecks failed while checking docker version https://review.openstack.org/544364 | 14:45 |
| paken | dtk: it went further now | 14:45 |
| paken | dtk: so right version is 2.7.0 | 14:46 |
| paken | dtk: so, in summary docker-py was released? | 14:47 |
| dtk | yup | 14:47 |
| dtk | due to the fact that they use the same namespace | 14:47 |
| dtk | and that the version was newer | 14:47 |
| dtk | guess what happened | 14:47 |
| dtk | heh | 14:47 |
| paken | ok, I understand now | 14:47 |
| paken | thanks | 14:47 |
| dtk | yw | 14:48 |
| paken | anyone updating the bug? | 14:48 |
| dtk | well, let me mark it as a duplicate | 14:54 |
| dtk | there was another one | 14:55 |
| paken | ok, thanks | 14:58 |
| *** coolsvap has joined #openstack-kolla | 15:02 | |
| -openstackstatus- NOTICE: Due to a race in stable/queens branch creation and some job removals, Zuul has reported syntax errors for the past hour; if you saw a syntax error reported for "Job tripleo-ci-centos-7-ovb-containers-oooq not defined" you can safely recheck now | 15:04 | |
| openstackgerrit | Kevin Tibi proposed openstack/kolla-ansible master: Add permission to notify dns server from worker https://review.openstack.org/538861 | 15:04 |
| *** paken has quit IRC | 15:08 | |
| *** mnasiadka has quit IRC | 15:12 | |
| *** gkadam has joined #openstack-kolla | 15:12 | |
| *** hamza21 has quit IRC | 15:18 | |
| *** lpetrut has quit IRC | 15:23 | |
| *** emccormick has joined #openstack-kolla | 15:28 | |
| *** dklyle has joined #openstack-kolla | 15:33 | |
| *** david-lyle has quit IRC | 15:33 | |
| *** k_mouza has quit IRC | 15:39 | |
| *** JamesBenson has joined #openstack-kolla | 15:41 | |
| inc0 | good morning | 15:45 |
| JamesBenson | morning inc0! | 15:45 |
| *** aikema has quit IRC | 15:46 | |
| mchlumsky | Hello. With regards to the tests failing in https://review.openstack.org/#/c/466674/ . I noticed that the globals.yml used for testing does not set keystone_token_provider to 'fernet' which will result in the keystone-ssh container never getting deployed. | 15:48 |
| *** itlinux has joined #openstack-kolla | 15:50 | |
| mchlumsky | Also, keystone fails to load the uuid keystone.token.provider, not sure if that's related to the missing keystone_token_provider in globals.yml but it should be loading the fernet provider. (http://logs.openstack.org/74/466674/5/check/kolla-ansible-centos-source/6fd7e4b/primary/logs/kolla/keystone/keystone.txt.gz) | 15:52 |
| *** sai___ has joined #openstack-kolla | 15:52 | |
| *** gkadam_ has joined #openstack-kolla | 15:52 | |
| *** aagate has joined #openstack-kolla | 15:53 | |
| *** wwriverrat_ has joined #openstack-kolla | 15:53 | |
| *** wwriverrat has quit IRC | 15:56 | |
| *** wwriverrat_ is now known as wwriverrat | 15:56 | |
| *** lvdombrkr89 has quit IRC | 15:58 | |
| emccormick | Anyone have a thought on why a server with no glance containers or images would pull both images but only create a container for glance-registry? | 15:58 |
| emccormick | I see it run a compare on both, but only recreate_or_restart_container on registry | 15:58 |
| pbourke | mchlumsky: yeah its a problem | 15:58 |
| pbourke | mchlumsky: ive added it as a topic for our meeting which is now in a few minutes | 15:58 |
| pbourke | emccormick: maybe an old api is stuck in restart? | 15:59 |
| *** rhallisey has quit IRC | 15:59 | |
| emccormick | running a deploy, it just skips the api container on the hosts it doesn't exist on yet | 15:59 |
| *** rhallisey has joined #openstack-kolla | 15:59 | |
| emccormick | doing a container ls -a there's nothing there | 15:59 |
| emccormick | would see it there, wouldn't I? | 16:00 |
| *** paken has joined #openstack-kolla | 16:00 | |
| pbourke | yeah you should do... | 16:00 |
| pbourke | maybe you can pastebin your deploy output? | 16:00 |
| pbourke | also make sure to run with -vvv | 16:01 |
| emccormick | It pulls the image down, but just never runs the create | 16:01 |
| emccormick | k | 16:01 |
| emccormick | will do. thanks. | 16:01 |
| *** egonzalez has quit IRC | 16:01 | |
| mchlumsky | pbourke, thank you, I will attend it | 16:01 |
| *** gkadam has quit IRC | 16:04 | |
| *** spiette_ has joined #openstack-kolla | 16:06 | |
| *** pcaruana has quit IRC | 16:13 | |
| ktibi | did anyone use kuryr on pike with kolla ? is it stable ? | 16:15 |
| *** kiennt26 has quit IRC | 16:20 | |
| *** mnasiadka has joined #openstack-kolla | 16:22 | |
| *** k_mouza has joined #openstack-kolla | 16:23 | |
| *** b_bezak has quit IRC | 16:24 | |
| *** b_bezak has joined #openstack-kolla | 16:24 | |
| *** k_mouza has quit IRC | 16:28 | |
| *** bmace has joined #openstack-kolla | 16:28 | |
| *** egonzalez has joined #openstack-kolla | 16:28 | |
| *** k_mouza has joined #openstack-kolla | 16:29 | |
| *** b_bezak has quit IRC | 16:29 | |
| emccormick | pbourke The log was too big even for pastebin pro | 16:33 |
| emccormick | https://1drv.ms/u/s!Au1QrS9ktpoLg37gDmT6x-Oi2luw | 16:33 |
| emccormick | pbourke You can look at it there in onedrive when you have a chance | 16:33 |
| emccormick | ktibi Think they're all in a meeting. That's all the help I can offer ;) | 16:34 |
| *** itlinux has quit IRC | 16:41 | |
| *** krtaylor has joined #openstack-kolla | 16:43 | |
| *** itlinux has joined #openstack-kolla | 16:52 | |
| *** mtsv has joined #openstack-kolla | 16:55 | |
| *** paken has quit IRC | 17:00 | |
| ktibi | inc0, how we vote for PTL ? | 17:01 |
| inc0 | ktibi: you should get email if you made commit during this cycle | 17:02 |
| *** lpetrut has joined #openstack-kolla | 17:03 | |
| ktibi | inc0, hum I'll check. I think I have commit in Q cycle. | 17:03 |
| *** egonzalez has quit IRC | 17:06 | |
| *** powerd has joined #openstack-kolla | 17:08 | |
| gkadam_ | inc0, i can see the current ptl election is for Rocky> | 17:10 |
| gkadam_ | Could you please help , how can I vote for the same? | 17:10 |
| gkadam_ | inc0, https://governance.openstack.org/election/#election-system , is this correct? | 17:11 |
| *** k_mouza has quit IRC | 17:12 | |
| *** k_mouza has joined #openstack-kolla | 17:13 | |
| inc0 | gkadam_: if you had commit in Q cycle, you'll get email with Rocky election | 17:14 |
| *** signed8bit is now known as signed8bit_Zzz | 17:15 | |
| gkadam_ | ah got it. | 17:15 |
| gkadam_ | inc0, thanks! | 17:15 |
| *** pcaruana has joined #openstack-kolla | 17:17 | |
| *** k_mouza has quit IRC | 17:19 | |
| *** devananda has joined #openstack-kolla | 17:24 | |
| *** mgoddard_ has quit IRC | 17:37 | |
| *** signed8bit_Zzz is now known as signed8bit | 17:41 | |
| *** dardelean has quit IRC | 17:41 | |
| *** dardelean has joined #openstack-kolla | 17:42 | |
| *** robbbe has quit IRC | 17:44 | |
| *** dardelean has quit IRC | 17:46 | |
| *** lpetrut has quit IRC | 17:48 | |
| *** awiddersheim has joined #openstack-kolla | 17:49 | |
| *** jmccarthy has quit IRC | 17:51 | |
| *** dardelean has joined #openstack-kolla | 17:52 | |
| *** hamza21 has joined #openstack-kolla | 17:52 | |
| *** lpetrut has joined #openstack-kolla | 17:56 | |
| *** dardelean has quit IRC | 17:56 | |
| *** k_mouza has joined #openstack-kolla | 17:56 | |
| *** sambetts is now known as sambetts|afk | 17:56 | |
| pbourke | it seems you also need to be registered with the foundation though which the above link doesn't make clear unfortunately | 17:57 |
| pbourke | emccormick: it looks like at line 13744 that it should be starting up glance-api on v69ctrl01 | 18:01 |
| *** k_mouza has quit IRC | 18:01 | |
| pbourke | odd | 18:01 |
| pbourke | inc0: should we actually merge this haproxy change given we're technically in feature freeze? | 18:02 |
| emccormick | pbourke well, it does have it on ctrl01. You'll see though that there's no create_or_restart event for ctrl02 and 03 | 18:04 |
| pbourke | ah ok that's something to go on | 18:04 |
| emccormick | pbourke: However you will see them for glance_registry | 18:04 |
| pbourke | let me check those nodes closer | 18:04 |
| emccormick | thanks! | 18:05 |
| *** signed8b_ has joined #openstack-kolla | 18:08 | |
| *** gfidente has quit IRC | 18:08 | |
| *** signed8bit has quit IRC | 18:11 | |
| pbourke | emccormick: lol | 18:14 |
| pbourke | emccormick: oh my | 18:14 |
| pbourke | emccormick: so this is actually a feature I implemented | 18:14 |
| emccormick | uhoh | 18:14 |
| pbourke | emccormick: wow | 18:14 |
| pbourke | emccormick: you're using just the file backend right? | 18:14 |
| emccormick | pbourke protecting me from myself? | 18:14 |
| pbourke | pretty much :) | 18:14 |
| emccormick | no | 18:14 |
| emccormick | ceph | 18:14 |
| emccormick | and file | 18:15 |
| emccormick | :D | 18:15 |
| pbourke | both? | 18:15 |
| emccormick | kind tricksy. There was migration going on from one to the other | 18:15 |
| emccormick | so they couldn't create in file anymore, but they could read them still | 18:15 |
| pbourke | ok well 99% whats happening is if you run the file backend for glance it will only start one api | 18:15 |
| emccormick | oooohhhhhh | 18:16 |
| pbourke | as the file backend isn't HA so images would end up on different apis | 18:16 |
| emccormick | that makes sense except there's a problem | 18:16 |
| pbourke | and nova would get confused and fail to find them | 18:16 |
| emccormick | file also is used for NFS | 18:16 |
| *** mgoddard_ has joined #openstack-kolla | 18:16 | |
| pbourke | it should check for that | 18:16 |
| pbourke | let me see | 18:16 |
| emccormick | it doesn't seem to be unless I accidentally removed something I shouldn't have | 18:16 |
| pbourke | if using NFS you need to set glance_file_datadir_volume | 18:17 |
| emccormick | or didn't put in a flag I should hae | 18:17 |
| pbourke | to the location of your share | 18:17 |
| emccormick | in globals? | 18:17 |
| pbourke | yes | 18:17 |
| pbourke | we probably need to make this more clear | 18:18 |
| emccormick | so it'll automount like cinder then? | 18:18 |
| emccormick | or I still need to mount it? | 18:18 |
| pbourke | well it will just bindmount that location into the container as opposed to using a docker volume | 18:18 |
| *** aagate has quit IRC | 18:23 | |
| emccormick | so by default it's set to "glance" | 18:24 |
| emccormick | so if I set it to my nfs volume (nas.vacld:/exports/openstack_images) that should allow it to go through | 18:24 |
| pbourke | so you should set glance_file_datadir_volume to /exports/openstack_images | 18:26 |
| pbourke | if Im remembering my nfs correctly | 18:26 |
| pbourke | basically whereever the export is on the filesystem | 18:26 |
| emccormick | so mount it somewhere, just not on the container volume like I used to | 18:27 |
| emccormick | and then point to the local path with glance_file_datadir_volum | 18:28 |
| *** signed8b_ has quit IRC | 18:32 | |
| *** signed8bit has joined #openstack-kolla | 18:33 | |
| *** openstackgerrit has quit IRC | 18:33 | |
| *** shardy has quit IRC | 18:38 | |
| *** unicell has joined #openstack-kolla | 18:56 | |
| *** mnasiadka has quit IRC | 18:56 | |
| *** powerd has quit IRC | 18:58 | |
| *** jbrooks has quit IRC | 19:03 | |
| *** jbrooks has joined #openstack-kolla | 19:05 | |
| *** mnasiadka has joined #openstack-kolla | 19:08 | |
| *** paken has joined #openstack-kolla | 19:08 | |
| *** k_mouza has joined #openstack-kolla | 19:09 | |
| *** lpetrut has quit IRC | 19:10 | |
| *** mnasiadka has quit IRC | 19:12 | |
| *** k_mouza has quit IRC | 19:13 | |
| *** billp has quit IRC | 19:14 | |
| inc0 | pbourke: sorry, no, you're right | 19:14 |
| inc0 | I jumped gun, this should go in after we branch | 19:14 |
| *** powerd has joined #openstack-kolla | 19:17 | |
| *** harlowja has joined #openstack-kolla | 19:21 | |
| *** aagate has joined #openstack-kolla | 19:21 | |
| *** mnasiadka has joined #openstack-kolla | 19:23 | |
| *** mnasiadka has quit IRC | 19:27 | |
| *** rwsu has quit IRC | 19:28 | |
| *** rwsu has joined #openstack-kolla | 19:29 | |
| *** dklyle has quit IRC | 19:33 | |
| *** mgoddard_ has quit IRC | 19:34 | |
| inc0 | kfox1111: around? | 19:34 |
| inc0 | or SamYaple | 19:34 |
| inc0 | or anyone else good at ceph:) | 19:35 |
| SamYaple | whats up? | 19:36 |
| inc0 | if you mount rbd to multiple hosts | 19:37 |
| inc0 | can you do it in multi-writer mode? | 19:37 |
| inc0 | assuming fs will handle it | 19:37 |
| SamYaple | nah | 19:38 |
| *** mnasiadka has joined #openstack-kolla | 19:38 | |
| SamYaple | i mean technically *probably* but there are a lot of write caches involved | 19:38 |
| SamYaple | so it wouldnt be in sync like that | 19:38 |
| inc0 | right | 19:38 |
| inc0 | alternative is cephfs but it's spof right? | 19:39 |
| bkero- | cephfs isn't spof in luminous aiui | 19:39 |
| SamYaple | nfs radosgw is a thing | 19:39 |
| SamYaple | cephfs is not spof in jewel either | 19:39 |
| SamYaple | its just not active/active | 19:39 |
| kfox1111 | inc0: hi. | 19:39 |
| SamYaple | its active/passive, but defeintely HA | 19:39 |
| kfox1111 | inc0: yes,no. | 19:39 |
| inc0 | ok, so barring performance, ceph fs is viable option | 19:40 |
| kfox1111 | I think you can mount it to multiple hosts, the same way you can with iscsi. | 19:40 |
| kfox1111 | but you need a filesystem on top of it to be multi mount safe. | 19:40 |
| inc0 | wasn't there like ceph lock that you had to lift for other node to write | 19:40 |
| inc0 | ? | 19:40 |
| kfox1111 | so, you can do it with read only ext4 for example. but if you want to write, you need a cluster file system like gpfs or oraclefs on top. which is not the best of things to do with ceph. | 19:40 |
| SamYaple | kfox1111: there is typically write caches involved that would still break that | 19:40 |
| SamYaple | i guess that could be configured around though | 19:41 |
| kfox1111 | right. | 19:41 |
| kfox1111 | but generally, not what you want. :) | 19:41 |
| inc0 | yeah I got that | 19:41 |
| inc0 | so cephfs would be best bet it sounds like | 19:41 |
| kfox1111 | yeah. you probably want cephfs. | 19:41 |
| kfox1111 | what are you trying to store? | 19:41 |
| *** coolsvap has quit IRC | 19:41 | |
| kfox1111 | because that can determine what you want to choose. | 19:42 |
| *** paken has quit IRC | 19:42 | |
| inc0 | ml stuff, some of it needs RW access | 19:42 |
| *** mnasiadka has quit IRC | 19:42 | |
| kfox1111 | ml? | 19:42 |
| kfox1111 | oh. machine learning? | 19:42 |
| SamYaple | inc0: radosgw has *native* nfs gateway | 19:42 |
| SamYaple | just fyi | 19:42 |
| SamYaple | http://docs.ceph.com/docs/master/radosgw/nfs/ | 19:43 |
| kfox1111 | that too. though kind of sketchy. :) | 19:43 |
| inc0 | yeah machine learning | 19:43 |
| kfox1111 | its not going to perform as well as cephfs. | 19:43 |
| kfox1111 | high io, or just occational for results? | 19:43 |
| SamYaple | not as well, but your client wont need kernel mods it doesnt have | 19:43 |
| SamYaple | toss up on when to use it | 19:43 |
| inc0 | kfox1111: way I understand just syncing stuff | 19:43 |
| inc0 | what needs performance can usually be RO so I'd use regular RBD for that | 19:44 |
| kfox1111 | SamYaple: true. though I think nfs-genesha with cephfs backend is still probably faster? havent tried though. | 19:44 |
| SamYaple | see the link kfox1111 | 19:44 |
| SamYaple | "Each NFS RGW instance is an NFS-Ganesha server instance embeddding a full Ceph RGW instance." | 19:44 |
| kfox1111 | heh. ah. so, same deal. :) | 19:44 |
| SamYaple | more or less, yea | 19:45 |
| SamYaple | performance should actually be a bit better | 19:45 |
| SamYaple | in some cases at least | 19:45 |
| SamYaple | none of this is good for lots of small file writes | 19:45 |
| inc0 | but cephfs probably will still outclass it right? | 19:45 |
| kfox1111 | could. it will be less posixy, so maybe higher performance. | 19:45 |
| kfox1111 | but, less posixy, so may break more. | 19:45 |
| SamYaple | inc0: native cephfs *might* outclass it | 19:45 |
| SamYaple | just dependso nthe workload | 19:45 |
| SamYaple | i would go native cephfs if its an option | 19:46 |
| kfox1111 | yeah. the way rados buckets files, a lot of little files will crush it much faster then with cephfs. | 19:46 |
| kfox1111 | which is still somewhat bad at it, but not as much. | 19:46 |
| SamYaple | ++ | 19:46 |
| inc0 | fortunately way I understand it is that acutally files are bigger, but fewer | 19:46 |
| kfox1111 | cephfs is easier to setup too IMO. | 19:46 |
| inc0 | so we're in better place | 19:46 |
| *** mnasiadka has joined #openstack-kolla | 19:52 | |
| inc0 | thanks for help kfox1111 SamYaple | 19:54 |
| kfox1111 | sure | 19:54 |
| SamYaple | always | 19:54 |
| SamYaple | kfox1111: you into golang? | 19:54 |
| kfox1111 | havent used it much in a while. | 19:55 |
| kfox1111 | in general, like it above a lot of other languages. | 19:55 |
| SamYaple | ah ok | 19:55 |
| inc0 | SamYaple: jascott1 is good resource for golang:) | 19:55 |
| kfox1111 | mostly converiting services to k8s these days. | 19:55 |
| SamYaple | i am writting the galera orchestrator in golang | 19:56 |
| SamYaple | inc0: yea be he isnt online :P | 19:56 |
| kfox1111 | SamYaple: awesome. | 19:56 |
| SamYaple | https://github.com/SamYaple/racimo/blob/master/cmd/racimo/main.go | 19:56 |
| inc0 | SamYaple: try slack | 19:56 |
| inc0 | he's on slack mostly | 19:56 |
| kfox1111 | been very interested in vitess too, as an alternate to galera. | 19:56 |
| SamYaple | yea ill do that | 19:56 |
| inc0 | helm-dev is where Justin lurks | 19:56 |
| SamYaple | yea i saw it, havent used it yet | 19:56 |
| kfox1111 | very curious if openstack would run well on it. | 19:56 |
| kfox1111 | I followed their tutorial and itseemed to work well. | 19:56 |
| inc0 | speaking of helm, I'm slowly playing with ksonnet | 19:57 |
| inc0 | it's quite cool too | 19:57 |
| kfox1111 | yeah. thats one of the things I'm going to discuss with the helm folks at the summit. pulling in some of that functionality. | 19:57 |
| *** mnasiadka has quit IRC | 19:57 | |
| kfox1111 | helm has some really great things, but some glaring holes in functionality too. | 19:57 |
| SamYaple | amen to that | 19:58 |
| kfox1111 | its okish if you own the whole stack of charts. its not so much otherwise. | 19:58 |
| *** rloo has joined #openstack-kolla | 20:05 | |
| *** mnasiadka has joined #openstack-kolla | 20:08 | |
| *** k_mouza has joined #openstack-kolla | 20:12 | |
| *** david-lyle has joined #openstack-kolla | 20:12 | |
| *** mnasiadka has quit IRC | 20:12 | |
| hamzy_ | if I run "sudo ./build.py --base centos --base-image centos --type source ceph" does that mean to build ceph from source? if so, then why does it yum install the ceph package at https://github.com/openstack/kolla/blob/master/docker/ceph/ceph-base/Dockerfile.j2#L14 | 20:15 |
| bmace | i'm just looking through the kolla-ansible plueprints in preparation for the ptg and it seems like they are super out of date? it is hard to tell what work got finished if any. i'm pretty sure the migration to ansible 2-2-0 was done because we are on 2.3 in pike | 20:15 |
| *** k_mouza has quit IRC | 20:16 | |
| *** mnasiadka has joined #openstack-kolla | 20:23 | |
| *** mnasiadka has quit IRC | 20:27 | |
| *** signed8bit is now known as signed8bit_Zzz | 20:28 | |
| *** signed8bit_Zzz is now known as signed8bit | 20:33 | |
| *** mnasiadka has joined #openstack-kolla | 20:38 | |
| *** mnasiadka has quit IRC | 20:43 | |
| *** dardelean has joined #openstack-kolla | 20:47 | |
| *** dardelean has quit IRC | 20:47 | |
| *** dardelean has joined #openstack-kolla | 20:47 | |
| *** k_mouza has joined #openstack-kolla | 20:49 | |
| *** mchlumsky_ has joined #openstack-kolla | 20:52 | |
| *** mnasiadka has joined #openstack-kolla | 20:53 | |
| *** k_mouza has quit IRC | 20:53 | |
| *** mchlumsky has quit IRC | 20:54 | |
| *** mnasiadka has quit IRC | 20:58 | |
| *** hamza21 has quit IRC | 21:06 | |
| *** mnasiadka has joined #openstack-kolla | 21:08 | |
| *** k_mouza has joined #openstack-kolla | 21:09 | |
| *** mnasiadka has quit IRC | 21:12 | |
| *** dardelean has quit IRC | 21:20 | |
| *** dardelean has joined #openstack-kolla | 21:21 | |
| *** inc0 has quit IRC | 21:21 | |
| *** inc0 has joined #openstack-kolla | 21:22 | |
| *** jtriley has quit IRC | 21:23 | |
| *** mnasiadka has joined #openstack-kolla | 21:23 | |
| *** jbadiapa has quit IRC | 21:24 | |
| *** dardelean has quit IRC | 21:26 | |
| *** mnasiadka has quit IRC | 21:27 | |
| *** pcaruana has quit IRC | 21:28 | |
| harlowja | has anyone tried using https://github.com/docker/docker-py/blob/master/Dockerfile to avoid a hypervisor (or other) even needing to have docker-py installed? | 21:30 |
| * harlowja was thinking that someone might of tried | 21:30 | |
| harlowja | (docker-py on centos brings in things and ... its ... not fun ...) | 21:31 |
| kfox1111 | in kolla-ansible? | 21:31 |
| harlowja | ya | 21:32 |
| harlowja | though not technically limited to that, but that's part of it, ha | 21:33 |
| kfox1111 | its a good idea, but no one has stepped up to do the work yet. | 21:33 |
| harlowja | SpamapS and i are working through this 'fun' | 21:33 |
| harlowja | kfox1111 kk, nobody has tried it that u know of, good to kno | 21:33 |
| kfox1111 | I wana do the same with kolla-kubernetes. | 21:33 |
| kfox1111 | no reason to need to install anything when you already have containers. | 21:33 |
| harlowja | ya, it'd be nice to fully isolate off docker-py and just have docker and that's it, lol | 21:33 |
| kfox1111 | yeah. | 21:34 |
| harlowja | in part cause https://github.com/docker/docker-py/blob/master/requirements.txt isn't that small (anymore, if it ever was) | 21:34 |
| harlowja | and its frozen versions, so i'm not really sure how flexible that version list is, lol | 21:35 |
| harlowja | though i guess https://github.com/docker/docker-py/blob/master/setup.py#L22-L51 is the unfrozen versions | 21:35 |
| *** powerd has quit IRC | 21:36 | |
| kfox1111 | I guess the rub for kolla-ansible is that you probably need it on the host to run a container? | 21:36 |
| kfox1111 | so chicken and the egg? | 21:36 |
| *** rhallisey has quit IRC | 21:36 | |
| kfox1111 | while, kolla-kubernetes relies on k8s, which already plumbs all the docker engines together into a cluster. | 21:36 |
| harlowja | ya, fair point, lol | 21:36 |
| *** k_mouza_ has joined #openstack-kolla | 21:37 | |
| kfox1111 | probably the best you could do then, is make all the bits you can into a virtualenv, and have ansile maintain that and run everything out of it? | 21:37 |
| kfox1111 | so making a psudo container. :/ | 21:37 |
| *** mnasiadka has joined #openstack-kolla | 21:38 | |
| kfox1111 | but, I guess the other way is to go ahead and run it in a container, use docker run to start it, but then exec in for doing anything. | 21:39 |
| kfox1111 | would still have the advantage of shipping all the files with docker, and little cli code. | 21:39 |
| *** ansiwen has quit IRC | 21:39 | |
| kfox1111 | but non zero. | 21:39 |
| *** k_mouza has quit IRC | 21:40 | |
| *** mnasiadka has quit IRC | 21:43 | |
| *** k_mouza has joined #openstack-kolla | 21:45 | |
| *** k_mouza_ has quit IRC | 21:48 | |
| harlowja | ya, apparently ansible can maybe be massaged to run things out of a venv | 21:49 |
| * harlowja hasn't tried that yet | 21:50 | |
| *** k_mouza has quit IRC | 21:50 | |
| kfox1111 | from what I remember of ansible, | 21:51 |
| kfox1111 | there was a two phase thingy, | 21:51 |
| *** mnasiadka has joined #openstack-kolla | 21:51 | |
| *** k_mouza has joined #openstack-kolla | 21:51 | |
| kfox1111 | where you could deploy a persistent daemon, then use that for all other runs. | 21:51 |
| kfox1111 | you could maybe tweak that to deploy the persistent daemon in a container, | 21:51 |
| harlowja | woah, lol | 21:51 |
| kfox1111 | then all subsequent runs would be inside. | 21:51 |
| kfox1111 | might work. | 21:52 |
| * kfox1111 shurgs | 21:52 | |
| kfox1111 | (or just help retool everyting on k8s. ;) | 21:52 |
| harlowja | momma | 21:52 |
| harlowja | lol | 21:52 |
| *** k_mouza has quit IRC | 21:52 | |
| *** k_mouza has joined #openstack-kolla | 21:53 | |
| *** k_mouza has quit IRC | 21:54 | |
| *** k_mouza has joined #openstack-kolla | 21:56 | |
| *** mnasiadka has quit IRC | 21:58 | |
| *** goldyfruit1 has joined #openstack-kolla | 22:00 | |
| *** goldyfruit has quit IRC | 22:01 | |
| *** mnasiadka has joined #openstack-kolla | 22:08 | |
| *** k_mouza has quit IRC | 22:11 | |
| *** k_mouza has joined #openstack-kolla | 22:12 | |
| *** mnasiadka has quit IRC | 22:13 | |
| *** k_mouza has quit IRC | 22:14 | |
| *** k_mouza_ has joined #openstack-kolla | 22:14 | |
| *** signed8bit is now known as signed8bit_Zzz | 22:16 | |
| *** signed8bit_Zzz is now known as signed8bit | 22:16 | |
| *** jtriley has joined #openstack-kolla | 22:16 | |
| *** ktibi has quit IRC | 22:18 | |
| *** dave-mccowan has quit IRC | 22:20 | |
| *** mnasiadka has joined #openstack-kolla | 22:23 | |
| *** dardelean has joined #openstack-kolla | 22:25 | |
| *** jtriley has quit IRC | 22:27 | |
| *** mnasiadka has quit IRC | 22:27 | |
| *** threestrands has joined #openstack-kolla | 22:31 | |
| *** mnasiadka has joined #openstack-kolla | 22:31 | |
| *** mnasiadka has quit IRC | 22:36 | |
| *** mnasiadka has joined #openstack-kolla | 22:38 | |
| *** mnasiadka has quit IRC | 22:43 | |
| *** itlinux has quit IRC | 22:53 | |
| *** mnasiadka has joined #openstack-kolla | 22:53 | |
| *** mnasiadka has quit IRC | 22:58 | |
| *** JamesBenson has quit IRC | 23:01 | |
| *** JamesBenson has joined #openstack-kolla | 23:03 | |
| *** JamesBenson has quit IRC | 23:07 | |
| *** mnasiadka has joined #openstack-kolla | 23:08 | |
| *** mnasiadka has quit IRC | 23:13 | |
| *** goldyfruit1 has quit IRC | 23:13 | |
| *** jtriley has joined #openstack-kolla | 23:21 | |
| *** mnasiadka has joined #openstack-kolla | 23:23 | |
| *** mnasiadka has quit IRC | 23:28 | |
| *** mnasiadka has joined #openstack-kolla | 23:38 | |
| *** mnasiadka has quit IRC | 23:43 | |
| *** signed8bit has quit IRC | 23:49 | |
| *** signed8bit has joined #openstack-kolla | 23:49 | |
| *** dardelean has quit IRC | 23:52 | |
| *** mnasiadka has joined #openstack-kolla | 23:53 | |
| *** mnasiadka has quit IRC | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!