| *** zhurong has quit IRC | 00:01 | |
| *** zhurong has joined #openstack-kolla | 00:02 | |
| sdake_ | Mech422 first off your not a random | 00:02 |
|---|---|---|
| sdake_ | Mech422 second off, only core reviewers may vote | 00:02 |
| *** zhurong has quit IRC | 00:04 | |
| *** bmace has quit IRC | 00:20 | |
| *** bmace has joined #openstack-kolla | 00:20 | |
| sdake_ | sup bmace | 00:31 |
| sbezverk | wirehead_ kfox1111: I tested splitted neutron bootstrap and it wokred without hicups.. I will run more tests, I think it should be the way to go until kube introduce a mechanism to run containers serially in the job.. | 00:32 |
| *** daneyon has joined #openstack-kolla | 00:33 | |
| *** daneyon has quit IRC | 00:38 | |
| *** duonghq has joined #openstack-kolla | 00:42 | |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP - l3 DaemonSets + DVR https://review.openstack.org/357557 | 00:48 |
| *** dave-mccowan has quit IRC | 00:58 | |
| bmace | hi sdake_ :) | 01:02 |
| *** caowei has joined #openstack-kolla | 01:04 | |
| *** eaguilar has quit IRC | 01:05 | |
| *** HyperJohnGraham has joined #openstack-kolla | 01:07 | |
| *** zhurong has joined #openstack-kolla | 01:09 | |
| *** kbaegis has quit IRC | 01:10 | |
| *** salv-orlando has joined #openstack-kolla | 01:15 | |
| *** salv-orlando has quit IRC | 01:15 | |
| *** salv-orlando has joined #openstack-kolla | 01:16 | |
| *** salv-orl_ has quit IRC | 01:17 | |
| *** salv-orlando has quit IRC | 01:21 | |
| *** unicell has quit IRC | 01:21 | |
| Pavo | anyone have an idea on how to fix this issue, this fix in this review does not work | 01:39 |
| Pavo | https://bugs.launchpad.net/kolla/+bug/1571612 | 01:39 |
| openstack | Launchpad bug 1571612 in kolla "Prechecks fails when checking if kolla_internal_vip_address is in the same network as network_interface on all nodes" [High,Triaged] - Assigned to Hiroki Ito (hrito) | 01:39 |
| openstackgerrit | Merged openstack/kolla: Customizations for Senlin https://review.openstack.org/352013 | 01:40 |
| duonghq | Pavo: which solution did you try? | 01:43 |
| Pavo | both | 01:43 |
| Pavo | in that review | 01:43 |
| Pavo | does not work | 01:43 |
| duonghq | Do you mean the comment? | 01:43 |
| Pavo | now trying this https://review.openstack.org/gitweb?p=openstack/kolla.git;a=commitdiff;h=7e00288ff09072eda41cc8a7daa53ac8bb94323f | 01:43 |
| duonghq | ah, ok, cannot see this link in bug page | 01:44 |
| Pavo | yeah tried changing /usr/share/kolla/ansible/roles/prechecks/task/ports_check.yml at the end of the file | 01:45 |
| *** rstarmer has joined #openstack-kolla | 01:45 | |
| Pavo | well second fix didn't work either | 01:46 |
| duonghq | are you deploying multinode? | 01:46 |
| Pavo | its the TASK: [prechecks | Checking if kolla_internal_vip_address is in the same network as network_interface on all nodes] part of the ports_check.yml file at the end | 01:46 |
| Pavo | yes | 01:46 |
| duonghq | (just for more information): which distro do you use and which user do you use for ssh? | 01:47 |
| Pavo | centos and root | 01:47 |
| duonghq | ah, ok | 01:47 |
| Pavo | I have ssh keys from deployer to all nodes | 01:48 |
| duonghq | sure :) | 01:48 |
| *** eaguilar has joined #openstack-kolla | 01:48 | |
| duonghq | I'm trying deploy multinode without remote root login | 01:49 |
| openstackgerrit | Merged openstack/kolla: Customizations for sahara https://review.openstack.org/346590 | 01:50 |
| openstackgerrit | Merged openstack/kolla: Update cirros image visibility in init-runonce https://review.openstack.org/353933 | 01:50 |
| duonghq | Pavo: can you try to debug with Ansible? | 01:50 |
| openstackgerrit | Merged openstack/kolla: Improve rabbitmq-clusterer template code format https://review.openstack.org/357065 | 01:50 |
| Pavo | how would I do that? | 01:51 |
| duonghq | we have friend here http://docs.ansible.com/ansible/debug_module.html | 01:52 |
| duonghq | and here | 01:52 |
| duonghq | #link https://docs.ansible.com/ansible/playbooks_debugger.html | 01:52 |
| Pavo | yeah I can look into that | 01:52 |
| Pavo | I am also having an issue when deploying it gets stuck at Starting Neutron Server | 01:56 |
| duonghq | hmm, mariadb container failed (again) | 02:00 |
| *** diogogmt has quit IRC | 02:00 | |
| Pavo | this is exactly whats happening to me https://ask.openstack.org/en/question/93376/during-kolla-deploy-when-neutron-comes-up-networking-goes-down/ | 02:00 |
| Pavo | no answers :( | 02:01 |
| duonghq | how many NIC do you have? | 02:01 |
| openstackgerrit | Merged openstack/kolla: Customizations for murano https://review.openstack.org/346340 | 02:01 |
| duonghq | Pavo: Do your mariadb keep restarting? | 02:02 |
| Pavo | 2 per node | 02:02 |
| Pavo | this is a network digram | 02:02 |
| Pavo | *diagram | 02:02 |
| openstackgerrit | Merged openstack/kolla: Create the _member_ role in the horizon role https://review.openstack.org/356082 | 02:03 |
| Pavo | https://www.lucidchart.com/invitations/accept/ddb42620-afc3-4489-aa63-a024034a190f | 02:03 |
| Pavo | eth0 is what neutron will use | 02:03 |
| duonghq | I don't have lucidchart acc | 02:04 |
| Pavo | this is my globals | 02:04 |
| Pavo | https://www.dropbox.com/s/2694rq4mkojt9ef/globals.yml?dl=0 | 02:04 |
| Pavo | oh ok hang on | 02:04 |
| sdake_ | pavo if stuck at starting neutron server, your interfaces are backwards | 02:04 |
| sdake_ | pavo though twe have been down this road already... | 02:05 |
| Pavo | yeah I thought so also | 02:05 |
| Pavo | doing exactly how you told me to | 02:05 |
| duonghq | anybody touch mariadb container recently? | 02:05 |
| Pavo | https://www.dropbox.com/s/n76ds55mzyyfeyt/Screen%20Shot%202016-08-18%20at%2010.05.31%20PM.png?dl=0 | 02:05 |
| Pavo | I don't understand why its failing | 02:06 |
| *** eaguilar has quit IRC | 02:06 | |
| openstackgerrit | Merged openstack/kolla: Customizations for zaqar https://review.openstack.org/346357 | 02:06 |
| Pavo | when it gets stuck on node1 which is control and network node I check the interfaces and eth0 and eth1 are up but docker0, ovs-system and br-ex are down | 02:08 |
| Pavo | nothing in logs and neutron-server container never gets started | 02:08 |
| openstackgerrit | Merged openstack/kolla: Fix publicURL endpoint not found issue in CI https://review.openstack.org/356513 | 02:08 |
| duonghq | Pavo: your config seems ok | 02:10 |
| Pavo | only thing I can think of is the I am using the wrong kolla_internal_vip_address | 02:10 |
| Pavo | if that IP is being used on the api interface like the docs say then that might be the issue | 02:11 |
| Pavo | docs say this | 02:11 |
| Pavo | http://pastebin.com/grgntY3U | 02:11 |
| duonghq | iirc, the api_interface is network_interface by default | 02:12 |
| duonghq | so, nothing wrong | 02:13 |
| *** huikang has joined #openstack-kolla | 02:13 | |
| Pavo | but the IP I am using is on a different network than the network_interface | 02:13 |
| Pavo | that IP in globals in on the eth0 network with is being used for neutron_external_interface | 02:14 |
| *** rstarmer has quit IRC | 02:15 | |
| duonghq | ah, miss reading, sorry | 02:16 |
| Pavo | gonna try changing that IP | 02:17 |
| Pavo | and use a different IP for kolla_external_vip_address | 02:18 |
| *** zhurong_ has joined #openstack-kolla | 02:19 | |
| *** dave-mccowan has joined #openstack-kolla | 02:19 | |
| *** zhurong has quit IRC | 02:20 | |
| duonghq | Pavo: do you ever get mariadb container keep restating? | 02:20 |
| Pavo | not yet | 02:20 |
| openstackgerrit | Merged openstack/kolla: Document the Dockerfile customisation mechanism https://review.openstack.org/351123 | 02:20 |
| Pavo | but I am also using stable | 02:20 |
| duonghq | ah, ok | 02:21 |
| duonghq | Pavo: mitaka? | 02:22 |
| Pavo | yeah | 02:22 |
| Pavo | 2.0.2 | 02:22 |
| duonghq | okay | 02:22 |
| *** rstarmer has joined #openstack-kolla | 02:24 | |
| openstackgerrit | Merged openstack/kolla: Update openvswitch base Dockerfile to include footers https://review.openstack.org/357243 | 02:24 |
| openstackgerrit | Merged openstack/kolla: Fix guide formating errors https://review.openstack.org/356235 | 02:24 |
| duonghq | sdake_: heavy lifting works :) tyvm | 02:25 |
| openstackgerrit | Merged openstack/kolla: Remove heat dev environment https://review.openstack.org/352119 | 02:25 |
| Pavo | guess I should make myself an account on launchpad so I can start submitting bugs I encounter | 02:30 |
| duonghq | Pavo: clearly | 02:30 |
| Pavo | lol funny | 02:30 |
| duonghq | welcome to the bug reporter team, lol | 02:31 |
| Pavo | lol | 02:31 |
| Pavo | I truly truly enjoy kolla's way of deploying openstack, coming from packstack thats saying alot | 02:31 |
| Pavo | went from Mirantis fuel to packstack now to kolla | 02:32 |
| Pavo | fuel is by far the worse in my book | 02:32 |
| duonghq | really? | 02:33 |
| *** huikang has quit IRC | 02:33 | |
| Pavo | yeah very proprietary doesn't fit our use case at all | 02:34 |
| duonghq | ah, | 02:35 |
| duonghq | your seems very interesting? | 02:35 |
| sdake_ | fuel is proprietary in what way? | 02:35 |
| Pavo | requires a minimum of 3 target nodes, best suited in enterprise environment | 02:35 |
| sdake_ | packstack had alot of potential shame red hat never turned it into their deployment system. They had a winner on their hands.. | 02:37 |
| Pavo | yeah our use case is VERY unique | 02:37 |
| Pavo | yeah I agree but packstack is very easy to deploy | 02:37 |
| Pavo | they switched focus to TripleO | 02:39 |
| Pavo | which is weird to me, reminds me of inception | 02:39 |
| mark-casey | Pavo you having maria issues? | 02:40 |
| mark-casey | just reading scrollback | 02:40 |
| Pavo | not yet | 02:40 |
| Pavo | duonghq was though I think | 02:41 |
| duonghq | Pavo: mark-casey | 02:41 |
| duonghq | yes, mariadb keep restarting | 02:42 |
| duonghq | thank Pavo for mentioned | 02:42 |
| Pavo | np | 02:42 |
| duonghq | trying to rebuild image | 02:44 |
| mark-casey | duonghq: do you have an error message/logs for the maria failure? | 02:45 |
| duonghq | I'm pull and reploying | 02:45 |
| duonghq | but with config do you use mark-casey | 02:45 |
| duonghq | *which | 02:46 |
| mark-casey | I've been working on a 3 node ubuntu source install | 02:48 |
| duonghq | I'm working on AIO, but get (probably) same issue | 02:48 |
| Pavo | ok here goes *crosses fingers* | 02:48 |
| mark-casey | but I use galera elsewhere so I thought I'd try to take a look if there was a maria issue | 02:48 |
| sdake_ | mark-casey - we had a report earlier thi sweek on osic not forming properly with galera | 02:49 |
| duonghq | in Kolla, Galera is always deployed, isn't it? | 02:49 |
| sdake_ | mark-casey don't know if its related - but it may be - did you try twice? | 02:49 |
| sdake_ | duonghq it can be turned off if you like | 02:49 |
| sdake_ | not sure how that would work in practice | 02:49 |
| *** rstarmer has quit IRC | 02:49 | |
| duonghq | sdake_: are you done with the commit review? | 02:50 |
| sdake_ | which one | 02:50 |
| sdake_ | all of them? | 02:50 |
| duonghq | yeah, I see that you review very much commit till now | 02:50 |
| sdake_ | ya taking a break | 02:50 |
| sdake_ | reviewing is really difficult work | 02:50 |
| sdake_ | requires very good attention to detail | 02:51 |
| sdake_ | which is not my strength :) | 02:51 |
| duonghq | tyvm sdake_ | 02:51 |
| mark-casey | sdake_: I'm not having a galera issue, just have some familiarity with it's sometimes-rough edges | 02:51 |
| *** huikang has joined #openstack-kolla | 02:52 | |
| duonghq | still do not understand why need twice deploy before bootstrap maria success | 02:52 |
| duonghq | but mariadb container keep failing and restarting | 02:53 |
| Mech422 | sdake_: thanks for the reply :-) Sounds like a good addition to the team... | 02:53 |
| duonghq | mark-casey: do you stuck here? | 02:53 |
| mark-casey | duonghq: no. I was asking about what you're seeing | 02:55 |
| duonghq | http://pastebin.com/1jLmn8w3 | 02:55 |
| duonghq | here | 02:55 |
| duonghq | mark-casey: please tell me if you have any idea | 02:56 |
| duonghq | thanks | 02:56 |
| *** yuanying has quit IRC | 02:56 | |
| duonghq | ah, Galera-related ones | 02:57 |
| *** yuanying has joined #openstack-kolla | 02:58 | |
| *** rstarmer has joined #openstack-kolla | 02:58 | |
| *** huikang has quit IRC | 03:02 | |
| mark-casey | duonghq: that is the container log, right? | 03:02 |
| duonghq | right | 03:02 |
| mark-casey | is this happening during deploy when bootstrapping from scratch? or upgrade/reconfigure? | 03:03 |
| duonghq | from scatch | 03:03 |
| mark-casey | does the kolla-ansible deploy fail the first time? | 03:03 |
| duonghq | yes | 03:03 |
| mark-casey | on which task? | 03:04 |
| duonghq | Running MariaDB bootstrap container | 03:04 |
| duonghq | after 2nd run without clean it pass this task | 03:04 |
| duonghq | but the mariadb cannot start | 03:05 |
| duonghq | it worked before | 03:06 |
| mark-casey | duonghq: distro? source/binary? | 03:06 |
| duonghq | host is trusty, image is ubuntu/source | 03:07 |
| duonghq | like you? | 03:07 |
| duonghq | mark-casey | 03:07 |
| mark-casey | duonghq: in the past yes. testing others now | 03:08 |
| mark-casey | duonghq: and you said this is aio? | 03:08 |
| duonghq | yup | 03:08 |
| openstackgerrit | Shaun Smekel proposed openstack/kolla: Add full support for fernet https://review.openstack.org/349366 | 03:12 |
| mark-casey | duonghq: can you kolla-ansible deploy with '-vvvv' and paste the first "Running MariaDB bootstrap container"? | 03:12 |
| duonghq | okay, hang on | 03:13 |
| *** coolsvap has joined #openstack-kolla | 03:13 | |
| duonghq | trying mariadb_recovery | 03:13 |
| duonghq | mark-casey: my log: http://pastebin.com/SzpwVwK6 | 03:18 |
| coolsvap | sdake_: around? | 03:18 |
| sdake_ | unfortuantely | 03:19 |
| sdake_ | what can i do for ya | 03:19 |
| *** salv-orlando has joined #openstack-kolla | 03:19 | |
| duonghq | mark-casey http://pastebin.com/RVTA73en | 03:20 |
| duonghq | boostrap log | 03:20 |
| coolsvap | sdake_: i didnt get the last message, should I expect a mail or something? | 03:20 |
| coolsvap | related to the status? | 03:20 |
| coolsvap | or I need to send? | 03:20 |
| *** ayoung has quit IRC | 03:22 | |
| *** yuanying has quit IRC | 03:22 | |
| *** huikang has joined #openstack-kolla | 03:22 | |
| sdake_ | coolsvap that was for informational purposes | 03:22 |
| duonghq | mark-casey: cannot jump in dead container to get more log | 03:23 |
| sdake_ | ttx said to forward on to other ptl candidates | 03:23 |
| sdake_ | hence why i ssent to you | 03:23 |
| mark-casey | duonghq: do you have deploy log like: | 03:24 |
| mark-casey | TASK [mariadb : Running MariaDB bootstrap container] *************************** | 03:24 |
| mark-casey | changed: [localhost] => {"changed": true, "invocation": {"module_args": {"action": "start_container", ... | 03:24 |
| mark-casey | probably just that task | 03:24 |
| coolsvap | sdake_: I think we are talking about two different things | 03:24 |
| duonghq | mark-casey: 1st or 2nd time run deploy? | 03:24 |
| coolsvap | I did not receive the mail you are talkin gabout | 03:25 |
| mark-casey | both if possible | 03:25 |
| coolsvap | I wwill start with osic based on pbourke last message | 03:25 |
| duonghq | mark-casey: 1st time is like I sent you | 03:25 |
| duonghq | in the end of log | 03:26 |
| mark-casey | duonghq: I see container logs but not kolla-ansible log | 03:27 |
| *** salv-orlando has quit IRC | 03:27 | |
| duonghq | http://pastebin.com/SzpwVwK6 | 03:27 |
| duonghq | mark-casey: http://pastebin.com/SzpwVwK6 I sent you both | 03:27 |
| duonghq | Pavo: any good news? | 03:28 |
| Pavo | not yet waiting for mariadb from VIP errored out | 03:29 |
| Pavo | changed IP in globals | 03:30 |
| Pavo | seems like it taking WAY long then it use to | 03:30 |
| duonghq | so you solved the VIP error? | 03:30 |
| Pavo | redeploying now, still waiting to get back to that spot | 03:30 |
| *** rstarmer has quit IRC | 03:30 | |
| duonghq | You don't have MariaDB error? | 03:30 |
| Pavo | this is the error I got | 03:31 |
| Pavo | http://pastebin.com/Li4MQYHK | 03:31 |
| duonghq | seems that haproxy-related | 03:32 |
| duonghq | ? | 03:32 |
| Pavo | maybe | 03:32 |
| duonghq | sdake_: are you there | 03:33 |
| *** huikang has quit IRC | 03:33 | |
| duonghq | Pavo: seem that MariaDB cause many troubles | 03:33 |
| sdake_ | coolsvap are you talking abou tthat mail from thierry i forwarded to you? | 03:33 |
| *** huikang has joined #openstack-kolla | 03:33 | |
| sdake_ | duonghq yes here | 03:33 |
| coolsvap | sdake_: yes I did not receive the mail you're talking about | 03:34 |
| duonghq | sdake_: can you take a look at my bp: https://blueprints.launchpad.net/kolla/+spec/ansible-specific-task-become | 03:34 |
| sdake_ | be with yo ugys in a bit | 03:35 |
| *** huikang has quit IRC | 03:38 | |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Add Dockerfile for congress https://review.openstack.org/305626 | 03:39 |
| *** huikang has joined #openstack-kolla | 03:40 | |
| Pavo | docker0 is DOWN, is that normal? | 03:44 |
| sdake_ | snafu | 03:44 |
| sdake_ | we use host networking | 03:45 |
| sdake_ | docker0 does nothing | 03:45 |
| Pavo | ok | 03:45 |
| Pavo | been TASK: [mariadb | Waiting for MariaDB service to be ready through VIP] ********* for about 10 minutes now | 03:46 |
| Pavo | container log shows 160818 23:36:37 [Note] WSREP: Service disconnected. | 03:47 |
| duonghq | how can I disable galera? | 03:48 |
| *** severion has quit IRC | 03:48 | |
| *** v1k0d3n has joined #openstack-kolla | 03:49 | |
| mark-casey | Pavo: could you paste docker logs for mariadb container for what it does when kolla-ansible is running "Running MariaDB bootstrap container"? | 03:50 |
| *** huikang has quit IRC | 03:50 | |
| Pavo | http://pastebin.com/D56M6jLA | 03:51 |
| *** v1k0d3n has quit IRC | 03:51 | |
| duonghq | Pavo: better than me | 03:52 |
| duonghq | no idea | 03:52 |
| Pavo | thats the entire mariadb log and still setting at Waiting for MariaDB service to be ready through VIP | 03:53 |
| Pavo | now it finally errored out | 03:53 |
| Pavo | Can't connect to MySQL server on '192.168.188.254 | 03:53 |
| *** yuanying has joined #openstack-kolla | 03:54 | |
| *** fragatina has quit IRC | 03:56 | |
| mark-casey | Pavo: that was from 'docker logs' for the mariadb container? any lines like "INFO:__main__:Setting permissions for /etc/mysql/my.cnf"? | 03:58 |
| duonghq | Pavo: reach same error as your | 03:58 |
| Pavo | not seeing any no | 03:59 |
| duonghq | Pavo: waiting for MariaDB... through VIP | 03:59 |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Add Dockerfile for congress https://review.openstack.org/305626 | 03:59 |
| Pavo | welcome to the waiting club lol | 04:00 |
| duonghq | it failed me quite fast :)) | 04:00 |
| Pavo | mine took forever | 04:01 |
| duonghq | ah, just my fault, | 04:01 |
| duonghq | lol | 04:01 |
| openstackgerrit | Merged openstack/kolla: Add force flag to cleanup-images https://review.openstack.org/356364 | 04:02 |
| Pavo | damn you | 04:02 |
| Pavo | lol | 04:02 |
| mdnadeem | sdake_, coolsvap , Jeffrey4l , can you please review patch set: https://review.openstack.org/#/c/349808/ https://review.openstack.org/#/c/349771/ | 04:02 |
| patchbot | mdnadeem: patch 349808 - kolla - Added Test case for env and volume comp | 04:02 |
| patchbot | mdnadeem: patch 349771 - kolla - Added Test case for attribute comp | 04:02 |
| openstackgerrit | Merged openstack/kolla: Fix the variable name mismatch https://review.openstack.org/349865 | 04:03 |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Added Ansible playbook for congress deploy https://review.openstack.org/350629 | 04:03 |
| sdake_ | sorry folks busy | 04:09 |
| Pavo | sdake_ have you tried out 10.12 yet? | 04:10 |
| Pavo | as in macOS 10.12 | 04:10 |
| Pavo | I mean | 04:10 |
| *** huikang has joined #openstack-kolla | 04:11 | |
| mark-casey | Pavo: duonghq: what was the docker logs command you used? | 04:14 |
| mark-casey | I think one must be mariadb and one bootstrap-mariadb | 04:14 |
| *** zhubingbing has joined #openstack-kolla | 04:14 | |
| Pavo | I did a less /var/lib/docker/volumes/kolla_logs/_data/mariadb/mairadb.log | 04:15 |
| duonghq | docker logs mariadb | 04:15 |
| mark-casey | oh cool. | 04:15 |
| Pavo | fixed my waiting for MariaDB service through VIP | 04:15 |
| mark-casey | what was it? | 04:15 |
| Pavo | enabled haproxy in globals lol | 04:15 |
| mark-casey | hehe | 04:15 |
| Pavo | forgot I disabled it | 04:16 |
| duonghq | lol | 04:16 |
| Pavo | fell like such an idoit | 04:16 |
| duonghq | but iirc it's enable by default | 04:16 |
| Pavo | feel | 04:17 |
| Pavo | not if you uncomment it lol | 04:17 |
| mark-casey | could you docker logs mariadb on the VM that ran "Running MariaDB bootstrap container" (if more than one node is running db) | 04:17 |
| Pavo | sure once this deployment either finishes or errors out | 04:17 |
| mark-casey | cool. thx | 04:17 |
| mark-casey | so duonghq... this is the task failing https://github.com/openstack/kolla/blob/master/ansible/roles/mariadb/tasks/bootstrap_cluster.yml#L2 on the first run right? | 04:18 |
| Pavo | just a fyi I am running stable version | 04:18 |
| Pavo | as in 2.0.2 | 04:18 |
| duonghq | mark-casey: yup, everytime after clean | 04:19 |
| mark-casey | it sets KOLLA_BOOTSTRAP: on line 8 to try and trigger this https://github.com/openstack/kolla/blob/master/docker/mariadb/extend_start.sh#L46 | 04:19 |
| mark-casey | which in turn calls "mysqld_safe --wsrep-new-cluster" | 04:20 |
| mark-casey | inside bootstrap_db | 04:20 |
| mark-casey | however... | 04:20 |
| *** rstarmer has joined #openstack-kolla | 04:20 | |
| mark-casey | the paste you sent (http://pastebin.com/1jLmn8w3) shows "Running command: '/usr/bin/mysqld_safe'" and "WSREP: Running position recovery" | 04:21 |
| mark-casey | where as Pavo's log shows "WSREP: 'wsrep-new-cluster' option used, bootstrapping the cluster" | 04:22 |
| mark-casey | I don't know why, but I think when your mariadb container bootstraps it is trying to restart a cluster, not start a new cluster | 04:22 |
| *** rstarmer has quit IRC | 04:22 | |
| Pavo | ok almost at the neutron server part, lets see if it gets stuck again | 04:22 |
| Pavo | *crosses fingers* | 04:23 |
| Pavo | woot it didn't get stuck | 04:24 |
| *** rstarmer has joined #openstack-kolla | 04:26 | |
| *** rstarmer has quit IRC | 04:27 | |
| mark-casey | hmm. duonghq can you try 'docker logs bootstrap-mariadb' during "Running MariaDB bootstrap container" task after clean? | 04:28 |
| *** rstarmer has joined #openstack-kolla | 04:29 | |
| *** rstarmer has quit IRC | 04:31 | |
| duonghq | mark-casey: I'll re deploy after check horizon if working | 04:31 |
| *** salv-orlando has joined #openstack-kolla | 04:32 | |
| mark-casey | cool. remember it may not work after task fails. I think the container will be removed | 04:32 |
| Pavo | why dos Gathering Facts take so long on stable | 04:33 |
| *** rstarmer has joined #openstack-kolla | 04:33 | |
| *** rstarmer has joined #openstack-kolla | 04:36 | |
| duonghq | horizon is working now | 04:37 |
| duonghq | cool | 04:37 |
| *** dims has quit IRC | 04:38 | |
| Pavo | nice | 04:38 |
| *** salv-orlando has quit IRC | 04:39 | |
| duonghq | mark-casey: the error disappear even after i clean container and host | 04:43 |
| duonghq | no idea | 04:43 |
| *** dims has joined #openstack-kolla | 04:43 | |
| mark-casey | not happening now/ | 04:44 |
| mark-casey | *? | 04:44 |
| duonghq | yup | 04:44 |
| mark-casey | very interesting. ok good :) | 04:44 |
| duonghq | I will record and pass you the log when it happen | 04:44 |
| duonghq | thank mark-casey | 04:45 |
| mark-casey | sure, happy to try | 04:45 |
| Pavo | I thought the kolla_external_vip_address was suppose to be added to the neutron_external_interface? | 04:48 |
| duonghq | it's reasonable to guess that | 04:50 |
| Pavo | yeah well I am done deploying lol but can't get to anything because all the IPs were added to the internal network lol | 04:50 |
| Pavo | not accessible from outside | 04:50 |
| Pavo | ok need some help here | 04:52 |
| Pavo | https://www.dropbox.com/s/n76ds55mzyyfeyt/Screen%20Shot%202016-08-18%20at%2010.05.31%20PM.png?dl=0 | 04:52 |
| Pavo | what interface would you use for network_interface and what interface would you use for neutron_external_interface | 04:53 |
| Pavo | eth1 and eth0 right | 04:53 |
| Pavo | ? | 04:53 |
| Pavo | the reason I ask is because this is what happened doing it that way | 04:56 |
| Pavo | http://pastebin.com/Awy4DLkH | 04:56 |
| Pavo | both the kolla_internal_vip_address and kolla_external_vip_address got set on eth1 | 04:57 |
| Pavo | which I thought was only for api interfaces | 04:57 |
| Pavo | which I thought was separated from all other traffic oh well, guess I am deplying again | 04:58 |
| Pavo | gonna do that tomorrow, cya | 05:05 |
| *** unicell has joined #openstack-kolla | 05:13 | |
| *** mark-casey has quit IRC | 05:13 | |
| *** dave-mccowan has quit IRC | 05:23 | |
| sdake_ | i'm off to bed | 05:23 |
| duonghq | bye sdake_ | 05:23 |
| sdake_ | night | 05:23 |
| sdake_ | 1 more day to go | 05:24 |
| duonghq | storm is coming | 05:24 |
| sdake_ | no doubt | 05:24 |
| *** sdake_ has quit IRC | 05:24 | |
| *** rstarmer has quit IRC | 05:30 | |
| *** rstarmer has joined #openstack-kolla | 05:31 | |
| openstackgerrit | Hui Kang proposed openstack/kolla: Add missing container image names to build config file https://review.openstack.org/356854 | 05:31 |
| *** rstarmer has quit IRC | 05:34 | |
| *** rstarmer has joined #openstack-kolla | 05:35 | |
| *** rstarmer has quit IRC | 05:37 | |
| *** salv-orlando has joined #openstack-kolla | 05:38 | |
| *** rstarmer has joined #openstack-kolla | 05:40 | |
| openstackgerrit | Hui Kang proposed openstack/kolla: Add missing container image names to build config file https://review.openstack.org/356854 | 05:41 |
| *** rstarmer has quit IRC | 05:43 | |
| *** huikang has quit IRC | 05:43 | |
| *** rstarmer has joined #openstack-kolla | 05:44 | |
| *** dmueller-evoila has joined #openstack-kolla | 05:48 | |
| *** rstarmer has quit IRC | 05:56 | |
| *** rstarmer has joined #openstack-kolla | 05:56 | |
| *** dmueller-evoila has quit IRC | 05:56 | |
| *** unicell has quit IRC | 05:57 | |
| *** unicell has joined #openstack-kolla | 05:57 | |
| *** daneyon has joined #openstack-kolla | 05:58 | |
| *** sdake has joined #openstack-kolla | 05:59 | |
| *** daneyon has quit IRC | 06:02 | |
| *** sdake has quit IRC | 06:08 | |
| *** zhubingbing_ has joined #openstack-kolla | 06:13 | |
| duonghq | hi zhubingbing_ | 06:14 |
| zhubingbing_ | hi | 06:14 |
| zhubingbing_ | duonghq heloo | 06:14 |
| zhubingbing_ | hello | 06:14 |
| duonghq | :D last working day of the week :D | 06:15 |
| *** fragatina has joined #openstack-kolla | 06:15 | |
| *** zhubingbing has quit IRC | 06:16 | |
| *** fragatina has quit IRC | 06:16 | |
| *** caowei has quit IRC | 06:17 | |
| *** fragatina has joined #openstack-kolla | 06:17 | |
| zhubingbing_ | yes，today is Friday | 06:18 |
| zhubingbing_ | Hui Kang | 06:19 |
| *** dmueller-evoila has joined #openstack-kolla | 06:23 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add gnocchi ansible role https://review.openstack.org/349351 | 06:28 |
| *** hieulq has quit IRC | 06:30 | |
| *** mdnadeem_ has joined #openstack-kolla | 06:30 | |
| *** dmueller-evoila has quit IRC | 06:32 | |
| *** hieulq has joined #openstack-kolla | 06:33 | |
| *** mdnadeem has quit IRC | 06:33 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair gnocchi dockerfile https://review.openstack.org/348388 | 06:33 |
| *** rstarmer has quit IRC | 06:34 | |
| *** rstarmer has joined #openstack-kolla | 06:35 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair aodh image dockerfile https://review.openstack.org/350715 | 06:38 |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair aodh image dockerfile https://review.openstack.org/350715 | 06:44 |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add aodh role https://review.openstack.org/351027 | 06:48 |
| *** Mr_Broken has joined #openstack-kolla | 06:51 | |
| zhubingbing_ | hi， coolsvap | 06:54 |
| coolsvap | zhubingbing_: hello | 06:54 |
| zhubingbing_ | i remove gnoochi/aodh user（in api），there changes are all in order to be able to run as httpd | 06:56 |
| zhubingbing_ | i refer to horizon and keystone. | 06:56 |
| coolsvap | zhubingbing_: sure thanks I have removed my -1 | 07:02 |
| *** cu5 has quit IRC | 07:09 | |
| zhubingbing_ | thanks | 07:13 |
| zhubingbing_ | thank you very much | 07:13 |
| *** salv-orl_ has joined #openstack-kolla | 07:16 | |
| *** salv-orlando has quit IRC | 07:19 | |
| *** salv-orl_ has quit IRC | 07:20 | |
| *** mnasiadka has joined #openstack-kolla | 07:21 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: fix sahara dockerfile https://review.openstack.org/351320 | 07:21 |
| *** cu5 has joined #openstack-kolla | 07:24 | |
| *** Serlex has joined #openstack-kolla | 07:31 | |
| *** rstarmer has quit IRC | 07:32 | |
| *** rstarmer has joined #openstack-kolla | 07:33 | |
| *** ad_rien_ has joined #openstack-kolla | 07:38 | |
| *** dmueller-evoila has joined #openstack-kolla | 07:41 | |
| *** daneyon has joined #openstack-kolla | 07:46 | |
| *** Mr_Broken has quit IRC | 07:47 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: fix sahara dockerfile https://review.openstack.org/351320 | 07:47 |
| *** Mr_Broken has joined #openstack-kolla | 07:47 | |
| *** daneyon has quit IRC | 07:51 | |
| *** Mr_Broken has quit IRC | 07:52 | |
| *** dmueller-evoila has quit IRC | 07:55 | |
| *** dmueller-evoila has joined #openstack-kolla | 07:55 | |
| openstackgerrit | Shaun Smekel proposed openstack/kolla: Add full support for fernet https://review.openstack.org/349366 | 07:56 |
| *** dmueller-evoila has quit IRC | 07:58 | |
| openstackgerrit | Shaun Smekel proposed openstack/kolla: Add dockerfiles for keystone fernet https://review.openstack.org/351139 | 07:59 |
| *** Mr_Broken has joined #openstack-kolla | 08:00 | |
| *** matrohon has joined #openstack-kolla | 08:01 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Sync the local_settings.j2 with horizon https://review.openstack.org/356471 | 08:01 |
| *** openstackgerrit has quit IRC | 08:03 | |
| *** openstackgerrit has joined #openstack-kolla | 08:03 | |
| *** joyee has joined #openstack-kolla | 08:09 | |
| *** duonghq has left #openstack-kolla | 08:14 | |
| *** gfidente has joined #openstack-kolla | 08:15 | |
| *** joyee has quit IRC | 08:22 | |
| *** Jeffrey4l has quit IRC | 08:27 | |
| *** athomas has joined #openstack-kolla | 08:29 | |
| *** Jeffrey4l has joined #openstack-kolla | 08:32 | |
| *** zhubingbing_ has quit IRC | 08:40 | |
| *** HyperJohnGraham has quit IRC | 08:49 | |
| Mr_Broken | Hey Guys | 09:05 |
| *** salv-orlando has joined #openstack-kolla | 09:06 | |
| Mr_Broken | deploying master/newton branch for the first time (ubuntu / binary ) | 09:06 |
| Mr_Broken | is this normal that some dockers dont build ? | 09:06 |
| Mr_Broken | http://paste.openstack.org/show/561159/ | 09:06 |
| Mr_Broken | I wanted to test out telemetry stuff (ceilometer + gnocchi) | 09:06 |
| Mr_Broken | but as you can see gnochi is building | 09:07 |
| pbourke | Mr_Broken: I think some of them are available for source builds only | 09:12 |
| pbourke | double check you should see something to that effect in the build output | 09:12 |
| Mr_Broken | yeah, noticed when tried to build only gnocchi from binary | 09:12 |
| Mr_Broken | there are no binaries yet | 09:13 |
| Mr_Broken | its building from sources now | 09:13 |
| Mr_Broken | thanks | 09:13 |
| *** egonzalez90 has joined #openstack-kolla | 09:16 | |
| pbourke | coolsvap: all rally tasks passing :) | 09:18 |
| pbourke | coolsvap: only ones failed seem to be cinder | 09:18 |
| pbourke | which is not deployed | 09:18 |
| coolsvap | pbourke: cool | 09:18 |
| pbourke | so lets sed in decent values now | 09:19 |
| pbourke | coolsvap: does concurrency of 2 and say times value of 5 mean it will do 10 tasks overall? | 09:19 |
| pbourke | coolsvap: or 5 tasks 2+2+1 | 09:19 |
| *** kbaegis has joined #openstack-kolla | 09:20 | |
| pbourke | actually it must be the second as otherwise you would have a lot more tasks showing in the list | 09:21 |
| pbourke | coolsvap: should we set the 'times' value to 100 or so? | 09:21 |
| coolsvap | pbourke: sorry was connecting to vpn, i think yes it would be 2nd | 09:22 |
| egonzalez90 | Jeffrey4l: ping | 09:25 |
| *** daneyon has joined #openstack-kolla | 09:35 | |
| Jeffrey4l | egonzalez90, pong | 09:35 |
| egonzalez90 | I've been thinking about removing the host list at https://review.openstack.org/#/c/355861/ | 09:36 |
| patchbot | egonzalez90: patch 355861 - kolla - Remove unnecessary host at haproxy in site.yml | 09:36 |
| egonzalez90 | gates are passing, but imho is because is an all in one, it will break in multimode deploys | 09:36 |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair aodh image dockerfile https://review.openstack.org/350715 | 09:36 |
| *** zhubingbing_ has joined #openstack-kolla | 09:36 | |
| zhubingbing_ | hello,coolsvap | 09:37 |
| egonzalez90 | because haproxy template need fact about other the other host | 09:37 |
| zhubingbing_ | I've already dealt with that problem. | 09:37 |
| zhubingbing_ | https://review.openstack.org/#/c/350715/ | 09:37 |
| patchbot | zhubingbing_: patch 350715 - kolla - repair aodh image dockerfile | 09:37 |
| coolsvap | zhubingbing_: alright I will wait for the the gate results | 09:37 |
| zhubingbing_ | ok,thanks | 09:38 |
| *** daneyon has quit IRC | 09:39 | |
| Jeffrey4l | egonzalez90, no check the haproxy/tasks/main.yml file | 09:39 |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Add basic checking for network_interface to prechecks https://review.openstack.org/355515 | 09:39 |
| egonzalez90 | I checked, but that defines where the role is applied. But https://github.com/openstack/kolla/blob/master/ansible/roles/haproxy/templates/haproxy.cfg.j2 need gather fact about the other hosts t get IP info | 09:41 |
| egonzalez90 | maybe I'm wrong, is just for ensure it will not break multinode | 09:41 |
| Jeffrey4l | egonzalez90, the facter is gather before the role is run. | 09:42 |
| Jeffrey4l | egonzalez90, it is not related. | 09:42 |
| egonzalez90 | ok, thanks | 09:42 |
| Jeffrey4l | egonzalez90, so i think leave only the haproxy is enough. | 09:43 |
| Jeffrey4l | but need eyes from other guys. | 09:44 |
| egonzalez90 | Jeffrey4l: thanks for clarify me on that | 09:45 |
| pbourke | coolsvap: can you let us know what values you think we need | 09:49 |
| coolsvap | pbourke: I think we can start with 100/500/1000 and concurrency level of 25/50/100 | 09:50 |
| pbourke | ok thanks | 09:50 |
| pbourke | coolsvap: replaced all values, can you have a quick look? | 10:02 |
| coolsvap | pbourke: just a min | 10:09 |
| coolsvap | pbourke: looks good | 10:10 |
| pbourke | coolsvap: ok think will go back and do the first deploy scenario | 10:11 |
| coolsvap | pbourke: yeah | 10:11 |
| Mr_Broken | I will never have a normal deploy: | 10:14 |
| Mr_Broken | http://paste.openstack.org/show/561232/ | 10:14 |
| Mr_Broken | anyone seen such a thing ? | 10:14 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Enable CADF events in Keystone and introduce kolla_enable_cadf_notifications https://review.openstack.org/349006 | 10:14 |
| Mr_Broken | looks like someone misplaced a quotation somewhere ? | 10:14 |
| pbourke | Mr_Broken: check your globals.yml | 10:15 |
| pbourke | for a backtick | 10:15 |
| Mr_Broken | why didnt i think of that one ? :] | 10:17 |
| Mr_Broken | <--- feels like a moron | 10:17 |
| Mr_Broken | thanks | 10:17 |
| *** iamkube has joined #openstack-kolla | 10:20 | |
| *** gbialas has joined #openstack-kolla | 10:21 | |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Add cloudkitty containers for type source https://review.openstack.org/349295 | 10:26 |
| *** gbialas has quit IRC | 10:29 | |
| *** psanchez has quit IRC | 10:29 | |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Add neutron-vpnaas-agent container https://review.openstack.org/335495 | 10:30 |
| *** psanchez has joined #openstack-kolla | 10:31 | |
| *** iamkube has quit IRC | 10:33 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Update Dockerfile to include footers https://review.openstack.org/357746 | 10:33 |
| *** athomas has quit IRC | 10:33 | |
| *** zhurong_ has quit IRC | 10:38 | |
| *** athomas has joined #openstack-kolla | 10:38 | |
| openstackgerrit | Merged openstack/kolla: Updated from global requirements https://review.openstack.org/357699 | 10:48 |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Handle empty package list for install_packages https://review.openstack.org/354429 | 10:49 |
| openstackgerrit | Merged openstack/kolla: Add Dockerfile for congress https://review.openstack.org/305626 | 10:51 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Add cloudkitty containers for type source https://review.openstack.org/349295 | 10:56 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Enable CADF events in Keystone and introduce kolla_enable_cadf_notifications https://review.openstack.org/349006 | 11:01 |
| *** Mr_Broken has quit IRC | 11:16 | |
| *** Mr_Broken has joined #openstack-kolla | 11:16 | |
| -openstackstatus- NOTICE: Precise tests on OSIC provider are currently failing, please stop your checks until the issue is resolved. | 11:16 | |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Customizations for Neutron https://review.openstack.org/350777 | 11:18 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Add cloudkitty containers for type source https://review.openstack.org/349295 | 11:20 |
| *** Mr_Broken has quit IRC | 11:20 | |
| -openstackstatus- NOTICE: DSVM jobs on OSIC currently failing because of IP collisions, fix is in the gate - https://review.openstack.org/#/c/357764/ - please hold rechecks until merged | 11:21 | |
| *** psanchez has quit IRC | 11:26 | |
| *** dmueller-evoila has joined #openstack-kolla | 11:26 | |
| *** psanchez has joined #openstack-kolla | 11:27 | |
| openstackgerrit | Md Nadeem proposed openstack/kolla: Added Ansible playbook for congress deploy https://review.openstack.org/350629 | 11:28 |
| *** dwalsh has joined #openstack-kolla | 11:30 | |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Add script to validate syntax of Jinja2 template files https://review.openstack.org/352378 | 11:31 |
| *** dmueller-evoila has quit IRC | 11:35 | |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Add neutron-vpnaas-agent container https://review.openstack.org/335495 | 11:35 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Integrate neutron-vpnaas-agent into the neutron Ansible role https://review.openstack.org/335502 | 11:35 |
| Daviey | pbourke: Hey. What is the current state of OSIC? | 11:37 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Add neutron-vpnaas-agent container https://review.openstack.org/335495 | 11:38 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Integrate neutron-vpnaas-agent into the neutron Ansible role https://review.openstack.org/335502 | 11:38 |
| *** ad_rien_ has quit IRC | 11:41 | |
| -openstackstatus- NOTICE: OSIC has burned through the problematic IP range with failures, things should be back to normal now. | 11:49 | |
| pbourke | Daviey: hey, rally scripts are ready now, so we're going back to the first deploy scenario and redoing | 11:54 |
| pbourke | Daviey: right now the destroy op seems to be hung though | 11:54 |
| pbourke | Daviey: if you have time could you take a look? | 11:54 |
| *** mnasiadka has quit IRC | 11:54 | |
| *** dave-mccowan has joined #openstack-kolla | 12:08 | |
| *** zhurong has joined #openstack-kolla | 12:16 | |
| Daviey | pbourke: sure | 12:22 |
| pbourke | Daviey: prechecks just ran through there | 12:22 |
| pbourke | Daviey: not sure whats wrong with the destroy | 12:22 |
| Daviey | pbourke: I bet my marbles it is ceph still mounted? | 12:23 |
| pbourke | Daviey: hmm | 12:24 |
| pbourke | Daviey: if prechecks pass we should be ok to deploy | 12:24 |
| Daviey | pbourke: wedged again? | 12:25 |
| Daviey | ah no | 12:25 |
| pbourke | Daviey: :/ | 12:28 |
| *** rhallisey has joined #openstack-kolla | 12:32 | |
| *** dmueller-evoila has joined #openstack-kolla | 12:34 | |
| *** Pavo has quit IRC | 12:34 | |
| *** sdake has joined #openstack-kolla | 12:35 | |
| *** rhallisey_ has joined #openstack-kolla | 12:35 | |
| sdake | morning | 12:35 |
| *** egonzalez90 has quit IRC | 12:36 | |
| *** zhubingbing_ has quit IRC | 12:36 | |
| *** zhubingbing_ has joined #openstack-kolla | 12:37 | |
| *** Pavo has joined #openstack-kolla | 12:38 | |
| Daviey | pbourke: Have we seen it wedge like that before? | 12:39 |
| pbourke | Daviey: no | 12:39 |
| Daviey | oh hm | 12:39 |
| *** rhallisey has quit IRC | 12:39 | |
| pbourke | Daviey: also dont know why passwords check is failing | 12:39 |
| pbourke | sigh | 12:39 |
| *** dmueller-evoila has quit IRC | 12:40 | |
| Daviey | pbourke: somehow nova key was "null" | 12:41 |
| Daviey | pbourke: regenen'd the passwords | 12:42 |
| *** rhallisey_ is now known as rhallisey | 12:47 | |
| sdake | heads up re summit: | 12:50 |
| sdake | Excellent question. The maximum number of Fishbowl+Workroom slots that | 12:50 |
| sdake | you can ask for in Barcelona is *16* (3 on Wednesday, 9 on Thursday, 4 | 12:50 |
| sdake | on Friday morning). | 12:50 |
| sdake | from ttx | 12:50 |
| sdake | another gem is that there are more teams to accomadate, less time in the summit schedule, and less overall capacity | 12:51 |
| *** dave-mcc_ has joined #openstack-kolla | 12:51 | |
| sdake | morning Daviey | 12:51 |
| sdake | moring dave-mcc_ | 12:51 |
| *** diogogmt has joined #openstack-kolla | 12:51 | |
| sdake | which means less slots | 12:51 |
| dave-mcc_ | good morning steve | 12:51 |
| sdake | we had the amximum allotment yesterday | 12:51 |
| sdake | sorry last summit | 12:51 |
| sdake | we may get trimmed considerably :) | 12:52 |
| sdake | especially since one of our slots went sort of wasted from wht I ouldd tell | 12:52 |
| *** mnasiadka has joined #openstack-kolla | 12:52 | |
| *** v1k0d3n has joined #openstack-kolla | 12:52 | |
| *** dave-mccowan has quit IRC | 12:53 | |
| Pavo | morning sdake | 12:53 |
| *** dave-mcc_ has quit IRC | 12:55 | |
| *** salv-orlando has quit IRC | 12:57 | |
| *** duonghq has joined #openstack-kolla | 13:03 | |
| pbourke | sdake: coolsvap: Jeffrey4l: can we merge this please https://review.openstack.org/#/c/354429/ | 13:03 |
| patchbot | pbourke: patch 354429 - kolla - Handle empty package list for install_packages | 13:03 |
| pbourke | otherwise need to keep updating as other customisation patches merge | 13:04 |
| sdake | i thought i did | 13:04 |
| sdake | maybe I just +2'ed it | 13:04 |
| *** Serlex has quit IRC | 13:04 | |
| sdake | oh i ran a recheck on it | 13:04 |
| sdake | because it was failing the gate | 13:04 |
| sdake | Jeffrey4l coolsvap can one of you ack that patch if it lgtu | 13:05 |
| sdake | rhallisey ping | 13:06 |
| pbourke | sdake: yeah it was failing as other patches merged | 13:06 |
| rhallisey | sdake, hey | 13:06 |
| coolsvap | sdake: I'm afk | 13:07 |
| pbourke | guys does anyone have the details of the networks we need to create post deploy | 13:08 |
| *** coolsvap is now known as _coolsvap_ | 13:08 | |
| pbourke | i thought we'd updated init-runonce but it seems no | 13:08 |
| pbourke | *not | 13:08 |
| *** Mr_Broken has joined #openstack-kolla | 13:08 | |
| sdake | _coolsvap_ roger - got your msg last night - ping me if when your ready | 13:08 |
| sdake | pbourke yes just cut and paste the init-runonce script | 13:09 |
| sdake | it creates a couple of things | 13:09 |
| pbourke | sdake: yeah but im not sure what needs to go in there | 13:09 |
| sdake | daneyon really worked most of that script out | 13:09 |
| pbourke | network ranges and the like | 13:09 |
| pbourke | are custom for osic | 13:09 |
| sdake | that is environmental | 13:09 |
| sdake | i hae no idea on that | 13:09 |
| sdake | there is a tenant network and a public floating ip network | 13:10 |
| sdake | both are needed | 13:10 |
| sdake | tenant network can be anything iirc | 13:10 |
| sdake | floating ip network must be something you can ssh into | 13:10 |
| sdake | be careful with tenant networking be anything part though - not sure if that would break others vlan's in osic | 13:10 |
| sdake | best to ping #osic | 13:11 |
| sdake | if you need immediate answer | 13:11 |
| *** Pavo has quit IRC | 13:12 | |
| duonghq | would you mind review my bp: https://blueprints.launchpad.net/kolla/+spec/ansible-specific-task-become | 13:12 |
| sdake | duonghq too late for this cycle unless you make it finished and merged prior to 31 | 13:13 |
| duonghq | sdake: of course not for this cycle | 13:14 |
| duonghq | so I should wait until next cycle and propose again? | 13:14 |
| sdake | duonghq if youcan get it done for this cycle i'd love to see that implemented | 13:14 |
| sdake | its a solid security feature | 13:14 |
| sdake | its marked as essential and approved | 13:14 |
| sdake | you pick which cycle yo uwant it to go into based upon that | 13:14 |
| sdake | but if not done by the 31st, it wont merge | 13:15 |
| *** jtriley has joined #openstack-kolla | 13:15 | |
| pbourke | duonghq: you should ping sean-k-mooney before starting on that also, as I believe he did some work there as part of the baremetal stuff | 13:15 |
| duonghq | sdake: thank you very much, IMO it's quite heavy work but will try my best | 13:15 |
| duonghq | pbourke: glad to here that | 13:16 |
| duonghq | thank you | 13:16 |
| sdake | note as long as the gates are green - even if it breaks something we can fix in the rc cycles | 13:16 |
| sdake | duonghq roger - wasn't aware it was alot of work | 13:16 |
| sdake | pleae fill in the work items field din the blueprint | 13:16 |
| sdake | i'll leave it untargeted to a specific release for now | 13:16 |
| duonghq | sdake: I'll start on basic block module than build up | 13:17 |
| sdake | rhallisey can you review pbourke 's patch aove | 13:17 |
| sdake | https://review.openstack.org/#/c/354429/ | 13:17 |
| patchbot | sdake: patch 354429 - kolla - Handle empty package list for install_packages | 13:17 |
| duonghq | sdake: iirc the Ubuntu gate is not stable yet? | 13:17 |
| sdake | duonghq right - its on my plate as soon as I have 5 minutes to think about it | 13:18 |
| pbourke | duonghq: it is not | 13:18 |
| openstackgerrit | Szymon Konefal proposed openstack/kolla: Remove Kazoo dependency and Kolla Zookeeper helper https://review.openstack.org/357866 | 13:19 |
| sdake | rhallisey it looks long and ominous but its 99% sed and 1 file of importance that needs reviewing | 13:20 |
| sdake | pbourke would you mind expanding on what you know of this ubuntu gate problem | 13:20 |
| sdake | you said keys are wrong? | 13:21 |
| sdake | do you have evidence of that ? | 13:21 |
| sdake | if you know where to look in the log file - it would help me out considierably :) | 13:21 |
| pbourke | sdake: the problems seem to be changing each time i look at it | 13:21 |
| sdake | probably infra working on it my guess | 13:22 |
| sdake | need to sync up with them on the mirroring if its a mirroring problem | 13:22 |
| pbourke | there was a key problem but im not seeing that on my latest patch | 13:22 |
| pbourke | so that may be fixed | 13:22 |
| sdake | if you ahve a link to a gate fialure recent one | 13:22 |
| pbourke | http://logs.openstack.org/29/354429/7/check/gate-kolla-dsvm-deploy-ubuntu-source-ubuntu-xenial-nv/ac3ca74/console.html#_2016-08-19_11_52_16_712892 | 13:22 |
| sdake | to the specific place it fails.. | 13:22 |
| sdake | ya infra may have fixed hte keys automagically | 13:23 |
| sdake | lots of people rely on the uca and ubuntu mirrors in openstack | 13:23 |
| sdake | so lots of folks pitch in to fix those and keep them operational | 13:23 |
| pbourke | we'll maybe hold off and see how persistent that mariadb issue is, those errors tend to fix after a while also | 13:26 |
| sdake | oh mysqldb module | 13:27 |
| * sdake groans | 13:27 | |
| sdake | ok well we can do an emergency fix there | 13:27 |
| sdake | followed by a proper fix in the rcs | 13:27 |
| sdake | to unblock the gate | 13:27 |
| sdake | that mysqldb module problem isn't going to fix itself magically | 13:27 |
| sdake | and i don't have ubuntu - but know 1 (incorrect) way to resolve it | 13:28 |
| pbourke | it might do, there was a similar neutron one other other day and by the time I had pushed a patch it had disappeared | 13:28 |
| sdake | and I really dont want to reload my boxes with ubuntu at this time | 13:28 |
| *** ayoung has joined #openstack-kolla | 13:28 | |
| sdake | was it mysqldb? | 13:28 |
| sdake | kolla-toolbox is a source-only container | 13:28 |
| pbourke | no it was a fixtures module | 13:28 |
| openstackgerrit | Merged openstack/kolla: Handle empty package list for install_packages https://review.openstack.org/354429 | 13:28 |
| pbourke | the packages churn on their dependencies | 13:28 |
| sdake | oh fixtures - ya dependencies like fixtures in neutron and friends always fix themselves over time | 13:29 |
| pbourke | thanks rhallisey | 13:29 |
| sdake | mariadb, not so much | 13:29 |
| pbourke | hmm | 13:29 |
| rhallisey | no problem | 13:29 |
| sdake | as you say they churn their deps | 13:29 |
| sdake | maraidb has much less dep churn | 13:29 |
| sdake | moment let me run a patch against the gate | 13:29 |
| sdake | bbiaf | 13:29 |
| pbourke | looking a bit closer this is ansible | 13:30 |
| pbourke | that should not have changed | 13:30 |
| sdake | its complaining about a missing dependnecy | 13:30 |
| sdake | mysqldb module not present | 13:30 |
| sdake | i'll sort it out today - focused effort | 13:31 |
| *** egonzalez90 has joined #openstack-kolla | 13:35 | |
| sdake | morning egonzalez90 | 13:35 |
| sdake | or evening as it may be ;) | 13:35 |
| egonzalez90 | evening :) | 13:35 |
| *** egonzalez90 has left #openstack-kolla | 13:35 | |
| *** egonzalez90 has joined #openstack-kolla | 13:36 | |
| sdake | man realestate in my area is off the hook pricewise | 13:36 |
| duonghq | evening | 13:36 |
| sdake | glad i bought when i did | 13:36 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Enable CADF events in Keystone and introduce kolla_enable_cadf_notifications https://review.openstack.org/349006 | 13:39 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Add cloudkitty containers for type source https://review.openstack.org/349295 | 13:42 |
| *** ad_rien_ has joined #openstack-kolla | 13:43 | |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Add script to validate syntax of Jinja2 template files https://review.openstack.org/352378 | 13:44 |
| sdake | akwasnie akwasnie_ ping | 13:47 |
| *** kolladude has joined #openstack-kolla | 13:51 | |
| sdake | hey look we have a kolladude ;) | 13:51 |
| Daviey | morning sdake o/ | 13:55 |
| duonghq | many kollish around | 13:56 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Create the nova compute instances when need during starting https://review.openstack.org/357898 | 13:57 |
| sdake | sup Daviey | 13:58 |
| *** eaguilar has joined #openstack-kolla | 14:00 | |
| *** eaguilar has quit IRC | 14:01 | |
| *** diogogmt has quit IRC | 14:03 | |
| *** diogogmt has joined #openstack-kolla | 14:04 | |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Enable CADF events in Keystone and introduce enable_cadf_notifications https://review.openstack.org/349006 | 14:05 |
| *** daneyon has joined #openstack-kolla | 14:05 | |
| *** mnasiadka has quit IRC | 14:05 | |
| *** eaguilar has joined #openstack-kolla | 14:07 | |
| sdake | wtb sean-k-mooney | 14:09 |
| sdake | anyone seen him as of late on irc? | 14:09 |
| *** huikang has joined #openstack-kolla | 14:10 | |
| wznoinsk | sdake: I'll ping him to drop in | 14:10 |
| *** daneyon has quit IRC | 14:10 | |
| sdake | wznoinsk thanks! | 14:10 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Implement mongodb replication set cluster https://review.openstack.org/345751 | 14:10 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Use dumb-init to manage the pid 1 process https://review.openstack.org/357015 | 14:14 |
| openstackgerrit | Merged openstack/kolla: Update heat base Dockerfile to include footers https://review.openstack.org/357240 | 14:14 |
| Jeffrey4l | Daviey, ping | 14:17 |
| Jeffrey4l | Daviey, need your feedback https://review.openstack.org/349764 | 14:17 |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Customizations for Nova https://review.openstack.org/351676 | 14:17 |
| *** ad_rien_ has quit IRC | 14:18 | |
| *** wznoinsk has quit IRC | 14:19 | |
| *** Pavo has joined #openstack-kolla | 14:21 | |
| *** sean-k-mooney has joined #openstack-kolla | 14:23 | |
| sean-k-mooney | o/ | 14:23 |
| sean-k-mooney | sdake where you looking for me a few minutes ago? | 14:24 |
| sdake | sean-k-mooney - sbezverk has a problem | 14:24 |
| sdake | sean-k-mooney I saw you sbumitted a review for neutron | 14:25 |
| *** JoseMello has joined #openstack-kolla | 14:25 | |
| sdake | sean-k-mooney any chance you can look at sbezverk's backtrace and provide guidance on how to resolve? | 14:25 |
| sdake | sean-k-mooney or find somone that can? | 14:25 |
| sean-k-mooney | i have one open for vhost-user reconnect. | 14:25 |
| sdake | I hae little contacts in neutron community | 14:25 |
| sean-k-mooney | sdake: sure i can take a look | 14:25 |
| sdake | and don't understand its implementation one bit | 14:25 |
| sbezverk | sean-k-mooney: I see some tracebacks generated by l3 agent | 14:27 |
| sdake | one traceback I thught | 14:27 |
| sbezverk | sdake: we started with one :-) | 14:27 |
| *** Mr_Broken has quit IRC | 14:27 | |
| sean-k-mooney | sbezverk: can you point me to them. im looking back at the irc logs now but i was not logged in when you strated talking about the issue | 14:27 |
| sdake | ok well sean-k-mooney can help you debug in his own style | 14:27 |
| *** Mr_Broken has joined #openstack-kolla | 14:28 | |
| sbezverk | sean-k-mooney: https://paste.fedoraproject.org/410484/47155530/ | 14:28 |
| sean-k-mooney | sbezverk: thanks ill take a look now | 14:29 |
| *** wznoinsk has joined #openstack-kolla | 14:30 | |
| sean-k-mooney | sbezverk: so first off i see an RTNETLINK answers: Invalid argument message. are you using the same host os as the container os? i havent read the full error yet but just taught i would ask | 14:30 |
| sdake | sean-k-mooney i picked upon that as well | 14:31 |
| sbezverk | sean-k-mooney: yes, host is cenots and container is centos | 14:31 |
| sdake | sean-k-mooney he is using k8s as an underlay however | 14:31 |
| sdake | sbezverk ar eyou running with host networking mode? | 14:31 |
| *** inc0 has joined #openstack-kolla | 14:32 | |
| *** Mr_Broken has quit IRC | 14:32 | |
| sdake | morning inc0 | 14:32 |
| sdake | how was diablo grinding :) | 14:32 |
| inc0 | I was drinking instead of diablo | 14:33 |
| inc0 | but today I may play a bit | 14:33 |
| sean-k-mooney | ok so if they are both centos it should not be a netlink format issue. not sure k8s should change the networking behavior in this way but perhaps it would. | 14:33 |
| sdake | so my wife showed me a picture of an alcohol isle in poland somewheerre | 14:33 |
| sdake | there were two full isles of vodka ;) | 14:33 |
| inc0 | isles of vodka? | 14:33 |
| sdake | isle = isle in grocery store | 14:33 |
| inc0 | ahh | 14:33 |
| inc0 | yeah, sounds about right | 14:34 |
| sdake | you guys mut really like your vodka :) | 14:34 |
| sean-k-mooney | sbezverk: what mount points are shared between the l3 agent continer and the host | 14:34 |
| sdake | sean-k-mooney hadn't thought of that.... | 14:34 |
| inc0 | I'm personally not a huge fan, but we have few good ones, yeah | 14:34 |
| *** eaguilar has quit IRC | 14:34 | |
| inc0 | Belvedere is best one you can buy around here | 14:34 |
| sdake | i like that vodka | 14:34 |
| inc0 | yeah, it's Polish | 14:35 |
| sdake | you can buy all kinds of fantastic vodka in the us | 14:35 |
| sean-k-mooney | sbezverk: sdake: if /sys is not shared it may not be able to read/edit the network namespaces | 14:35 |
| sbezverk | sean-k-mooney: /run and /run/netns | 14:35 |
| inc0 | this is mainstream I know is here | 14:35 |
| inc0 | havent seen much more | 14:35 |
| inc0 | also, I'm not exactly a specialist in this field | 14:35 |
| duonghq | sdake: I draft some work items, but need to breakdown some item in times https://blueprints.launchpad.net/kolla/+spec/ansible-specific-task-become | 14:35 |
| sean-k-mooney | sbezverk: ok let me check if we share anyting else in the ansible deployment | 14:35 |
| sdake | but you have to go to speciality store | 14:35 |
| *** michauds has joined #openstack-kolla | 14:36 | |
| sdake | duonghq thanks :) | 14:36 |
| NachoDuck | Hi all, I've been trying to get our Kolla eval to connect to our external network on a vlan provider but it doesn't seem to work. I've added the custom config in ml2_config.ini from the Red Hat docs (seems to be the most complete docs I've found) but I still can't ping an instance that has been assigned an IP from our local range, is there anything Kolla | 14:36 |
| NachoDuck | specific I might have missed for this? | 14:36 |
| duonghq | sdake: I think n-3 is achivable, will try my best | 14:36 |
| sbezverk | sean-k-mooney: but we do not have /sys in kolla | 14:36 |
| sdake | duonghq - just so i know, are you working on any other current kolla related work? | 14:36 |
| sdake | duonghq want to understand if that other work is higher prioirty | 14:36 |
| sbezverk | sean-k-mooney: basically we mimic all shares from kolla in kube | 14:36 |
| sdake | sbezverk docker reporting the shares via docker inspect? | 14:37 |
| sdake | maybe your specifying shares, but the shares raen't happening | 14:37 |
| duonghq | sdake: I also work on swift-kubernetes, but I think this bp is more important, so I will balance my time, | 14:37 |
| sdake | duonghq definatley more important (new bp) | 14:37 |
| inc0 | NachoDuck, soooo...we debugged tons of it in our osic cluster, but at the end it worked without any issue with kolla, so I can confirm it does work | 14:37 |
| duonghq | sdake: (the ansible is more important, I mean) | 14:37 |
| duonghq | sdake: agree | 14:37 |
| sdake | duonghq agree 100% | 14:37 |
| inc0 | so, what's your configs? | 14:37 |
| sdake | i'd say not even balance at this point | 14:37 |
| *** zhubingbing_ has quit IRC | 14:38 | |
| duonghq | sdake: much more effort on ansbile, of course | 14:38 |
| sdake | focus on become localization | 14:38 |
| sdake | anyway up to you | 14:38 |
| sdake | do what you like :) | 14:38 |
| sdake | just wanted to make sure something critical wasn't being dropped in favor | 14:38 |
| sdake | (critical in the next 2 weeks) | 14:38 |
| sdake | kubernetes-swift can wait awhile | 14:39 |
| duonghq | sdake: thanks, I will do ansible first, absolutely | 14:39 |
| *** zhubingbing_ has joined #openstack-kolla | 14:39 | |
| sdake | duonghq we need compute kit oeprational first on k8s | 14:39 |
| NachoDuck | inc0: Alright well that's encouraging at least cheers. I didn't have to change much, here's what I added for the ml2_config - https://gist.github.com/anonymous/4d34064fcc832ebdc215258d69211d43 | 14:39 |
| sdake | duonghq if you prefer to work on k8s, may i suggest focusing on getting nova going there | 14:39 |
| sdake | duonghq get a vm to boot is the objective | 14:39 |
| sean-k-mooney | sbezverk: with the k8s deployment can you still docker exec into the continer? | 14:39 |
| inc0 | NachoDuck, just one tag? ok, how about your globals.yml? | 14:39 |
| sdake | NachoDuck kolla community provides basic support on getting up and running | 14:40 |
| sean-k-mooney | sbezverk: im wondering if you can run the failing command youreslf and see if there is any other output? | 14:40 |
| duonghq | sdake: due to nova and neutron is on good proress, it is take time to catch up with them, so I will do the ansible become first | 14:40 |
| duonghq | sdake: do you agree with me? | 14:40 |
| sdake | but we don't have a ton of operator experience on our team, so figuring out each environment for yu is not really our cup of tea | 14:40 |
| sdake | it is hard for us to help there is what I mean to say :) | 14:40 |
| sdake | duonghq agree | 14:40 |
| sdake | neutron is covered | 14:40 |
| sdake | nova is not | 14:40 |
| sdake | that may change by time become is reoslved | 14:41 |
| NachoDuck | inc0: Yep at the moment that's it. globals - https://gist.github.com/anonymous/8363ebfecbcf0b5bdcb6a899bc9493dc | 14:41 |
| duonghq | sdake: hope that, thank you | 14:41 |
| sdake | NachoDuck but the architecture and implementation have been proved at 100 node scale | 14:41 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP - l3/metadata DaemonSets + DVR https://review.openstack.org/357557 | 14:41 |
| NachoDuck | sdake: Yeah I realise that's the case, I don't expect you to debug the entire thing but if its just a case of "yep what you've got should work" then maybe I can did a big deeper | 14:41 |
| sdake | with ssh into vms and all the other things | 14:42 |
| sbezverk | sean-k-mooney: I will give it ia try, for now I am back to fight with mutual hateness between kube and ovs | 14:42 |
| sdake | NachoDuck one thing you might have setup wrong is your neutron netowrking after deployment | 14:42 |
| sdake | NachoDuck - there is a developer script in the repo called init-runonce | 14:42 |
| sdake | NachoDuck worth referencing - but recommend not using unless you tailor to your environment | 14:42 |
| sean-k-mooney | sbezverk: well the error look like the network namespace is not visable from within the l3 agent contaienr | 14:43 |
| sdake | sbezverk i'd double check that docker inspect on the nodes shows the shares you think it should be showing | 14:43 |
| sdake | sean-k-mooney thanks for the help :) | 14:43 |
| NachoDuck | sdake: Alright noted, I have run that after my deployments just to get the images setup etc but it doesn't include an example of a vlan provider iirc? | 14:43 |
| sdake | NachoDuck no idea what it does - daneyon wrote it | 14:44 |
| sbezverk | sean-k-mooney: thank you for your help, so far your are the only one from neutron who even tried to help us :-) | 14:44 |
| inc0 | NachoDuck, and em2 switch is configured for trunk right? | 14:44 |
| sdake | NachoDuck all I know is it works in my environment ;) | 14:44 |
| sdake | so i checked it in to the repo | 14:44 |
| sdake | NachoDuck i am not a networking expert | 14:44 |
| sdake | and as my wife says "Your company didn't build their empire because networking is easy." | 14:44 |
| NachoDuck | inc0: Well the vlan I'm trying is the native vlan for that em2 port, so while I realise that might be a bit weird, it should work right? | 14:45 |
| sean-k-mooney | sbezverk: no worries. i have worked on nova and neutron for almost the last 3 years so if i can help im glad too | 14:45 |
| NachoDuck | sdake: Yeah, no worries, I know some of the basics but by no means and expert either | 14:46 |
| inc0 | NachoDuck, if it's native, try using flat network rather than vlan | 14:47 |
| sean-k-mooney | sbezverk: if ip netns shows the router namespace can you do a ip netns qrouter-cd1706eb-47a3-4496-83bc-8b5bef26a533 bash and let me know if it works | 14:48 |
| NachoDuck | inc0: Gotcha I'll give it a go | 14:48 |
| inc0 | I'm not sure if tagged traffic can even work on non-trunked switchport | 14:49 |
| egonzalez90 | +1 to inc0 if is native you have to use flat networks as the interface doesn't know that a vlan exists | 14:49 |
| NachoDuck | Right makes sense, well TIL. Cheers guys | 14:49 |
| inc0 | no problem NachoDuck let us know if it worked | 14:50 |
| duonghq | sdake: can you set the milestones of ansbile bp to n-3, quite tight schedule but achivable | 14:51 |
| sdake | duonghq you got it | 14:52 |
| sdake | duonghq if it gets into danger territory let me know | 14:53 |
| sdake | ink to bp? | 14:53 |
| duonghq | sdake: sure | 14:54 |
| sdake | ok sean-k-mooney if you need to depart irc to focus on something else - understand - thanks for dropping in quickly :) | 14:55 |
| sbezverk | sean-k-mooney: I tried that and got something like invalid namespace | 14:55 |
| sbezverk | sean-k-mooney: ip netns exec qrouter-120a14d1-c943-4a4e-ab2b-a045d6386665 ifconfig | 14:56 |
| sbezverk | setting the network namespace "qrouter-120a14d1-c943-4a4e-ab2b-a045d6386665" failed: Invalid argument | 14:56 |
| *** HyperJohnGraham has joined #openstack-kolla | 14:58 | |
| NachoDuck | Yep that works, cheers again :) | 15:00 |
| Pavo | can I specify what interface the kolla_external_vip_address goes on? | 15:00 |
| *** salv-orlando has joined #openstack-kolla | 15:01 | |
| *** lrensing_ has joined #openstack-kolla | 15:01 | |
| sdake | Pavo - kolla_external_vip_address is for tls - are you using tls? | 15:01 |
| sdake | if not, leave it commented out | 15:01 |
| Pavo | roger | 15:01 |
| inc0 | NachoDuck, glad I could help | 15:01 |
| Pavo | so... kolla_internal_vip_address gets set on the network_interface which by default is the api_interface correct? | 15:02 |
| openstackgerrit | Hui Kang proposed openstack/kolla: Add missing container image names to build config file https://review.openstack.org/356854 | 15:03 |
| Pavo | can I set the api_interface to be different from the network_interface? | 15:03 |
| *** zhurong has quit IRC | 15:03 | |
| inc0 | Pavo, it should, but never tried that;) | 15:03 |
| inc0 | it has to be in same network tho | 15:03 |
| Pavo | ok | 15:04 |
| inc0 | well...hmm...maybe even not | 15:06 |
| openstackgerrit | Merged openstack/kolla: Start using orchestration_engine variable https://review.openstack.org/356538 | 15:06 |
| inc0 | as long as it's routable | 15:06 |
| openstackgerrit | Abishek Subramanian proposed openstack/kolla: Fix the location of Kolla configurations in the evaluation workflow https://review.openstack.org/357520 | 15:06 |
| lrensing_ | hey guys, how would i go about fixing an error during kolla-ansible deploy:neutron : Waiting the openvswitch_db service to be ready | 15:07 |
| sean-k-mooney | sbezverk: interesting so did qrouter-120a14d1-c943-4a4e-ab2b-a045d6386665 was visable when you did ip netns but ip netns exec failed | 15:07 |
| *** salv-orlando has quit IRC | 15:08 | |
| inc0 | lrensing_, docker logs openvswitch_db | 15:08 |
| openstackgerrit | Szymon Konefal proposed openstack/kolla: Remove Kazoo dependency and Kolla Zookeeper helper https://review.openstack.org/357866 | 15:09 |
| sean-k-mooney | sbezverk: one think i noticed on my ubuntu 16.04 system is that network nampseaces now show up in mount. http://paste.openstack.org/show/561354/ | 15:10 |
| sean-k-mooney | sbezverk: do you see the namespaces mounted in /run/netns/ | 15:11 |
| sbezverk | sean-k-mooney: My setup is not running atm, since I started chasing another issue. I will double check when I recover it.. | 15:17 |
| kfox1111 | in a jinja2 template, whats the easiest way to get the first defined var? | 15:17 |
| sean-k-mooney | sbezverk: ok no worries. the command that the agent is running look correct so my assumtion is that for some reason the info cannont be read withing the continer. | 15:18 |
| *** Serlex has joined #openstack-kolla | 15:18 | |
| sean-k-mooney | sbezverk: either /run is not mapped shared, or something else in the container enviroment is preventing it from working | 15:19 |
| sean-k-mooney | sbezverk: from the error though im not sure what it is currently | 15:19 |
| *** vbel has quit IRC | 15:20 | |
| *** vbel has joined #openstack-kolla | 15:21 | |
| inc0 | pbourke, damn | 15:29 |
| inc0 | you guys merged too fast coolswaps reviews;) | 15:30 |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair gnocchi dockerfile https://review.openstack.org/348388 | 15:30 |
| kfox1111 | ah. got it. |default() | 15:33 |
| *** haplo37__ has joined #openstack-kolla | 15:34 | |
| *** mark-casey has joined #openstack-kolla | 15:36 | |
| Pavo | ok here are all the cmds I ran to get the target nodes ready for deployment, someone mind seeing if I am missing anything http://pastebin.com/fZeGub9e | 15:36 |
| *** daneyon_ has quit IRC | 15:36 | |
| Pavo | for CentOS btw | 15:37 |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add gnocchi ansible role https://review.openstack.org/349351 | 15:38 |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair aodh image dockerfile https://review.openstack.org/350715 | 15:38 |
| *** ayoung has quit IRC | 15:39 | |
| openstackgerrit | Merged openstack/kolla: Customizations for Elasticsearch https://review.openstack.org/354508 | 15:41 |
| *** _coolsvap_ is now known as coolsvap | 15:42 | |
| coolsvap | inc0: any issues | 15:43 |
| inc0 | coolsvap, sooo | 15:44 |
| inc0 | yeah | 15:44 |
| zhubingbing_ | hello | 15:44 |
| inc0 | we can't ahve footers in base images | 15:44 |
| inc0 | that will make build under proxy impossible | 15:44 |
| coolsvap | inc0: i was under the same impression | 15:44 |
| sdake | sup folks | 15:44 |
| coolsvap | and i -1ed 3-4 reviews | 15:44 |
| coolsvap | but Jeffrey4l and pbourke convinced me that we need it | 15:45 |
| inc0 | ehh | 15:45 |
| zhubingbing_ | I want to know how this string can be obtained, implement-mongo-replicate-set-cluster-0d3f140f7116c3ba.yaml | 15:45 |
| zhubingbing_ | 0d3f140f7116c3ba | 15:45 |
| sdake | inc0 - we have 1 week left of access to osic, is that correct? | 15:45 |
| *** salv-orlando has joined #openstack-kolla | 15:45 | |
| Pavo | sdake does this look right for getting target host ready for deployment for CentOS http://pastebin.com/fZeGub9e | 15:45 |
| Jeffrey4l | coolsvap, what? | 15:45 |
| *** duonghq has quit IRC | 15:46 | |
| coolsvap | Jeffrey4l: the footer part in customization | 15:46 |
| irtermite | sdake: til 8/30 | 15:46 |
| inc0 | initial reason we have header and footer was to enable people to put ENV http_proxy=whatevere and in footer ENV no_proxy=my-networks | 15:46 |
| sdake | irtermite thanks! | 15:46 |
| Jeffrey4l | coolsvap, i think it is find. when need some thing like: i want to add a package in all nova related image. | 15:46 |
| irtermite | and don't go over or I'll brake your knee caps for it back ;) | 15:46 |
| Jeffrey4l | s/find/fine | 15:46 |
| kfox1111 | 1;3D | 15:47 |
| inc0 | Jeffrey4l, that's different thing | 15:47 |
| irtermite | just don't tell anyone I gave you 4 instead of 3 weeks, sdake ;) | 15:47 |
| inc0 | and will be done with customizations | 15:47 |
| mark-casey | rofl... that escalated quickly | 15:47 |
| inc0 | and I'm ok with having named blocks in dockerfiles | 15:47 |
| inc0 | but not anonymus footer | 15:47 |
| Daviey | inc0: TBH, i didn't bother adding proxy to the header.. I just use http_proxy in the environment before executing kolla-build | 15:48 |
| Jeffrey4l | inc0, OK. but something like that, I want to add one file to the all nova images? | 15:49 |
| Jeffrey4l | Daviey, need your feedback https://review.openstack.org/349764 | 15:49 |
| Daviey | Looking | 15:49 |
| sdake | irtermite - i tell noone - unless its already in our docs :( | 15:49 |
| Daviey | Anyone on OSIC atm? | 15:49 |
| irtermite | hahaha is ok... not a big deal sdake | 15:50 |
| inc0 | Daviey, that didn't work before, maybe docker fixed this | 15:50 |
| coolsvap | Daviey: I am not | 15:50 |
| Daviey | Jeffrey4l: Sorry for the delay in responding to that.... I'll see if we can get any traction in #openstack-nova now? | 15:50 |
| inc0 | Jeffrey4l, so with new customizations, you need to use customization file with nova_base_packages_append=[my_packages] | 15:51 |
| Jeffrey4l | Daviey, np. I am in now. | 15:51 |
| sdake | mark-casey - which escalated quickly | 15:51 |
| inc0 | also if you'd just use --append-footer it would add it to every containers | 15:51 |
| Jeffrey4l | inc0, my bad example. | 15:51 |
| sdake | mark-casey you mean the osic discussion above, or the dockerfile customizations? | 15:52 |
| inc0 | Jeffrey4l, https://github.com/openstack/kolla/blob/master/doc/image-building.rst#package-customisation | 15:52 |
| inc0 | also we can add named blocks, as I said | 15:52 |
| inc0 | but not block footer | 15:52 |
| inc0 | we can add block nova-base-footer | 15:53 |
| openstackgerrit | Merged openstack/kolla: Add basic checking for network_interface to prechecks https://review.openstack.org/355515 | 15:53 |
| mark-casey | sdake: just irtermite's knee cap joke | 15:53 |
| *** daneyon has joined #openstack-kolla | 15:54 | |
| openstackgerrit | Merged openstack/kolla: Optimize horizon cache and compress for http response https://review.openstack.org/346444 | 15:54 |
| Jeffrey4l | inc0, hmm. wait. what we are talk? nova-base-footer block i think... | 15:54 |
| inc0 | in coolsvap | 15:55 |
| inc0 | coolsvap's review, it's also block footer | 15:55 |
| inc0 | so maybe I panicked;) | 15:55 |
| irtermite | hahha mark-casey | 15:55 |
| inc0 | I'll make -1 out of -2, just removing block footer should be ok | 15:55 |
| Jeffrey4l | inc0, coolsvap could we talk base on a code example? | 15:55 |
| inc0 | https://review.openstack.org/#/c/357245/2 | 15:56 |
| patchbot | inc0: patch 357245 - kolla - Update ceilometer base Dockerfile to include footers | 15:56 |
| coolsvap | inc0: i will take care of it | 15:57 |
| coolsvap | will update the patch | 15:57 |
| coolsvap | and rest of things which are merged | 15:57 |
| inc0 | kk I'll scan our code to catch footers in base | 15:57 |
| inc0 | ok, you can do it if you want | 15:57 |
| inc0 | I'll remove -2 | 15:57 |
| Jeffrey4l | my bad. the ceilomter_base_footer is find. but the footer block is not. | 15:58 |
| *** daneyon has quit IRC | 15:58 | |
| inc0 | yeah, ceilometer_base_footer is ok | 15:58 |
| *** matrohon has quit IRC | 16:02 | |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add gnocchi ansible role https://review.openstack.org/349351 | 16:03 |
| openstackgerrit | zhubingbing proposed openstack/kolla: repair gnocchi dockerfile https://review.openstack.org/348388 | 16:03 |
| egonzalez90 | coolsvap: include senlin-base at https://review.openstack.org/#/c/357746/, block footers were merged today | 16:04 |
| patchbot | egonzalez90: patch 357746 - kolla - Update Dockerfile to include footers | 16:04 |
| coolsvap | egonzalez90: sure | 16:05 |
| coolsvap | I willcreate single patch | 16:05 |
| egonzalez90 | cool | 16:05 |
| coolsvap | or probably update the last one | 16:05 |
| Daviey | Jeffrey4l: FWIW, Kolla exposed a race condition in live migrations a few weeks ago, based on all the services starting concurrently | 16:06 |
| Daviey | Jeffrey4l: they fixed it pretty quickly | 16:06 |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add gnocchi ansible role https://review.openstack.org/349351 | 16:07 |
| Jeffrey4l | Daviey, so? | 16:07 |
| Daviey | Jeffrey4l: Just saying that #openstack-nova were pretty receptive to helping solve issues previously | 16:08 |
| Daviey | Jeffrey4l: But not very responsive on a Friday evening i guess | 16:08 |
| Daviey | (no response yet) | 16:08 |
| Jeffrey4l | Daviey, yep they are. | 16:08 |
| openstackgerrit | zhubingbing proposed openstack/kolla: Add aodh role https://review.openstack.org/351027 | 16:09 |
| *** egonzalez90 has quit IRC | 16:14 | |
| Daviey | Jeffrey4l: If we get no response, i'll kick off a thread on the ML on Monday | 16:16 |
| *** Serlex has quit IRC | 16:17 | |
| Jeffrey4l | Daviey, OK. cool. | 16:17 |
| openstackgerrit | Kevin Fox proposed openstack/kolla-kubernetes: WIP - l3/metadata DaemonSets + DVR, & labels https://review.openstack.org/357557 | 16:18 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Optimize horizon cache and compress for http response https://review.openstack.org/357996 | 16:19 |
| *** coolsvap is now known as _coolsvap_ | 16:20 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Use a random virtual_router_id in the keepalived https://review.openstack.org/357997 | 16:20 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Use a random virtual_router_id in the keepalived https://review.openstack.org/357998 | 16:21 |
| *** fragatina has quit IRC | 16:22 | |
| *** zhubingbing_ has quit IRC | 16:25 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Fix the kolla_docker issue with docker 1.12 https://review.openstack.org/358002 | 16:26 |
| kfox1111 | does neutron-server actually talk to ovs? I didn't think it did. | 16:28 |
| kfox1111 | I thought it talked to neutron-openvswitch-agent which did. | 16:28 |
| inc0 | kfox1111, or lbaas right? | 16:42 |
| openstackgerrit | Christian Berendt proposed openstack/kolla: Improve script to check used source versions https://review.openstack.org/352640 | 16:42 |
| kfox1111 | yeah. | 16:42 |
| kfox1111 | I'm wondering why there is an effort to expose ovs via tcp. | 16:43 |
| kfox1111 | if it works with containers now, why does it need to be opened up to work with kolla-kubernetes? it shoudl work the same I think. | 16:43 |
| inc0 | that would remove need of having ovs agent on cnode | 16:43 |
| kfox1111 | ovs is needed on the network nodes. | 16:44 |
| kfox1111 | but neutron-server can run without it I think. | 16:44 |
| *** ayoung has joined #openstack-kolla | 16:44 | |
| kfox1111 | neutron-server shouldn't ever be talking directly to ovs. | 16:44 |
| *** fragatina has joined #openstack-kolla | 16:45 | |
| sbezverk | kfox1111: it does not | 16:47 |
| sbezverk | kfox1111: all goes via neutron-openvswitch-agent | 16:47 |
| sbezverk | kfox1111: according to Jeffrey4l accessing ovs via tcp is default way and it is more efficient than via socket | 16:48 |
| kfox1111 | sbezverk: can you give this a try please: http://efox.cc/temp/hello-initcontainer.yaml | 16:48 |
| *** athomas has quit IRC | 16:49 | |
| sbezverk | also the issue with socket is it gets created by root, but neutron-openvswitch-agent runs as neutron, so it fails accessing this socket | 16:49 |
| kfox1111 | sbezverk: default for what? | 16:49 |
| sbezverk | for openstack | 16:49 |
| kfox1111 | ... | 16:49 |
| * kfox1111 goes and checks his systems... | 16:49 | |
| kfox1111 | its not default, at least in rdo. | 16:51 |
| kfox1111 | socket permissions, I could see being an issue. | 16:51 |
| kfox1111 | somehow rdo solved it though. | 16:51 |
| *** fragatina has quit IRC | 16:51 | |
| *** fragatina has joined #openstack-kolla | 16:52 | |
| sbezverk | kfox1111: that is why I proposed first to run as root, but kolla cores refused, second option was to switch to tcp port | 16:52 |
| kfox1111 | my neutron-openvswitch-agent is also running as neutron. | 16:52 |
| sbezverk | btw your example with init container worked | 16:53 |
| sbezverk | kfox1111: can you check permissions on ovs dn socket? | 16:54 |
| kfox1111 | know off hand where it is? | 16:54 |
| kfox1111 | sbezverk: awesome. :) | 16:54 |
| *** srwilkers_ has joined #openstack-kolla | 16:54 | |
| kfox1111 | got a 10:00. so gota leave in a sec. | 16:55 |
| kfox1111 | but somehow rdo's working with a very similar ovs setup. | 16:55 |
| kfox1111 | ls -l /var/run/openvswitch/*sock /var/run/openvswitch/*.ctl | 16:56 |
| kfox1111 | srwx------ 1 root root 0 Apr 19 18:48 /var/run/openvswitch/db.sock | 16:56 |
| kfox1111 | srwx------ 1 root root 0 Apr 19 18:48 /var/run/openvswitch/ovsdb-server.1220.ctl | 16:56 |
| kfox1111 | srwx------ 1 root root 0 Apr 19 18:48 /var/run/openvswitch/ovs-vswitchd.1270.ctl | 16:56 |
| kfox1111 | neutron has User=neutron in systemd's unit. | 16:56 |
| sbezverk | kfox1111: can you explain how neutron-openvswitch can access this socket without any permissions ;-) | 16:56 |
| kfox1111 | so its never starting as root. | 16:56 |
| sbezverk | kfox1111: I bet it goes over tcp port | 16:57 |
| *** sould has joined #openstack-kolla | 16:57 | |
| kfox1111 | don't know. no ovs stuff is showing up in netstat as listening on tcp. | 16:58 |
| kfox1111 | I'll look some more after the meeting. | 16:58 |
| sdake | uid of ovsdb is what? | 16:58 |
| kfox1111 | root. | 16:59 |
| sdake | why not change permission on the socket to group kolla? | 16:59 |
| sdake | surely that is configurable | 16:59 |
| sdake | sbezverk ^^ | 16:59 |
| *** srwilkers_ has quit IRC | 17:00 | |
| sdake | rather owner of socket | 17:00 |
| sdake | and permission to 770 | 17:00 |
| sdake | even if it is not configurable it can be changed after startup with a sudo operation | 17:00 |
| sbezverk | sdake socket gets created when we start ovsdb process, I do not see where we can change | 17:01 |
| *** dmueller-evoila has joined #openstack-kolla | 17:01 | |
| sbezverk | this permissions, I tried to change it manually whiule the process was running ir did not help | 17:01 |
| sdake | after ovsdb socket is created | 17:01 |
| sdake | which permissions did yo uuse? | 17:02 |
| sdake | sbezverk is this the root cause of your segfault? | 17:02 |
| kfox1111 | sdake: for our system, we use openvswitch for the root networking too. | 17:02 |
| Pavo | OMG finally have a complete deployment | 17:02 |
| kfox1111 | we have 1 40gig connection coming in for the system, then break it out with vlans. | 17:02 |
| sdake | pavo grats! | 17:02 |
| kfox1111 | so kolla can't own openvswitch on our system. | 17:03 |
| sdake | kfox1111 inereting | 17:03 |
| sdake | kfox1111 not sure how to add that in at the moment | 17:03 |
| kfox1111 | el7 supports ovs in its native networkstack. | 17:03 |
| kfox1111 | we can just skip the launching of the ovs containers I think. | 17:03 |
| sdake | that may do it | 17:03 |
| kfox1111 | sbezverk: is it a namespace issue of some kind maybe? | 17:05 |
| sdake | _coolsvap_ i responded on that vote thread so you could see the email now | 17:05 |
| kfox1111 | oh.... | 17:06 |
| kfox1111 | maybe it is a namespace thing. | 17:06 |
| kfox1111 | sbezverk: have you tried: hostIPC=true > | 17:06 |
| kfox1111 | ? | 17:06 |
| sbezverk | sdake: no, as I mentioned there are multiple issue with ovs in kube it is just one of them.. | 17:06 |
| kfox1111 | otherwise the container has its own socket namespace. | 17:07 |
| sdake | oh ya | 17:07 |
| sbezverk | kfox1111: usually with ipc I would see process gets stuck, but I can try | 17:07 |
| sdake | wonder if thats a bug in ansible code as well | 17:07 |
| sdake | sbezverk kfox1111 suggested using host ipc rather then container namespaced ipc | 17:08 |
| sdake | lthough I have only found this to be a problem on abstract sockets | 17:08 |
| sbezverk | kfox1111: IMHO the issue is kube starting by some unknown to me reason two identical pod on the same compute node, 1 pod stays 2nd goes away after a bit of time, but since they start at the same time they cause conflicts to each other | 17:09 |
| sdake | as long as /run is host mounted | 17:09 |
| Pavo | surprising running actually well in VMs | 17:09 |
| sdake | pavo as it should :) | 17:09 |
| sdake | Pavo once you sort out your vm networking config | 17:09 |
| Pavo | finally, after 4 days | 17:09 |
| sbezverk | kfox1111: they try to access the same socket, open the same port | 17:10 |
| Pavo | so... I had to put both interfaces on the same network | 17:10 |
| sdake | pavo fwiw - if you used bare metal would likely take 1-2 hours ;) | 17:10 |
| Pavo | yeah gonna try it out on 3 HP servers with 96 CPUS, 512Gb RAM and 24Tb of storage | 17:11 |
| Pavo | wanna try out all services on allnodes | 17:11 |
| sdake | pavo some services probably work better then others | 17:11 |
| *** lrensing_ has quit IRC | 17:11 | |
| Pavo | in the inventory file there is nothing gonna go wrong if I add all nodes to all roles right? | 17:12 |
| Pavo | testing out reconfigure now | 17:14 |
| sdake | reconfigure and adding nodes = not tested | 17:14 |
| sdake | but i think it shold work | 17:14 |
| inc0 | 512 Gig of RAM...you sir have a monster. | 17:15 |
| Pavo | yeah have 3 of them identical | 17:15 |
| Pavo | all of them have max cpus and RAM and 24Tb storage | 17:15 |
| Pavo | so reconfigure it used for adding nodes correct? | 17:16 |
| sdake | pavo deploy is used to add nodes | 17:16 |
| sdake | deploy is idempotent | 17:16 |
| Pavo | ah | 17:16 |
| inc0 | it's like Daenerys and her 3 dragons:P (btw I just compared server hardware to fantasy dragons...you don't get more nerdy than that) | 17:16 |
| sdake | reconfigure might get you there | 17:16 |
| sdake | not sure | 17:16 |
| inc0 | anyway, that's an idea for hostnames isn't it? | 17:17 |
| Pavo | inc0 you know this is the last season of GoT coming correct? | 17:17 |
| Pavo | and its only gonna have 7 episodes :( | 17:17 |
| inc0 | yeah | 17:17 |
| Pavo | very sad | 17:18 |
| sould | very sad indeed | 17:18 |
| inc0 | well, I've been through many endings like that that left me half-empty | 17:18 |
| inc0 | Witcher 3 was the last one | 17:18 |
| sould | the reimagined battlestar series left me feeling that way | 17:18 |
| inc0 | battlestar was another. | 17:19 |
| Pavo | yeah that ending was weird | 17:19 |
| *** dwalsh has quit IRC | 17:19 | |
| inc0 | it was kinda far fetched | 17:19 |
| inc0 | but it was cool | 17:20 |
| Pavo | yeah, seemed like it rushed | 17:21 |
| *** gfhellma has joined #openstack-kolla | 17:22 | |
| inc0 | gfhellma, Gustavo?:) welcome | 17:22 |
| sdake | battlestar galactica season 3 could have been skipped and right on to seson 4 | 17:22 |
| sould | agreed | 17:23 |
| sdake | the reason it seemed rushed is the writers didn't know what to do | 17:23 |
| inc0 | sdake, not really, whole thing about Baltars trial | 17:23 |
| sdake | becaue it was hard to top season 1 and 2 | 17:23 |
| gfhellma | inc0, good to be here! Thanks for the pointer! :) | 17:23 |
| sdake | the right thing to do there would be to finish on a high note with season 4 :) | 17:23 |
| sould | that was around the time of the writers strike right? | 17:23 |
| sould | i think thats why it went on hiatus for a bit with everything else | 17:23 |
| sdake | dont recall details | 17:24 |
| sdake | that was years ago | 17:24 |
| sdake | that was just the ipression it left me with :) | 17:24 |
| inc0 | I kinda liked season 3 with all the post-new-caprica aftermath | 17:24 |
| inc0 | anyway, all the good things has to end before they become bad things | 17:25 |
| Pavo | true | 17:26 |
| inc0 | that being said, I sometimes come back to OST from battlestar | 17:27 |
| inc0 | I thing it is brilliant | 17:27 |
| Pavo | OST? | 17:27 |
| inc0 | soundtrack | 17:27 |
| Pavo | oh | 17:27 |
| sould | the rendition of all along the watchtower gives me chills -- i like it | 17:28 |
| inc0 | like their version of All Along The Watchtower | 17:28 |
| sbezverk | kfox1111: ping | 17:28 |
| inc0 | sould, exactly. | 17:28 |
| sould | :D | 17:28 |
| inc0 | listening to it right now btw | 17:28 |
| sould | i just queued it up on spotify | 17:28 |
| inc0 | the scene it was soundtrack was brilliant too | 17:29 |
| inc0 | I think this was conclusion of season 4, when Kara went to the vortex | 17:29 |
| kbaegis | They should've ended it a few episodes early | 17:30 |
| kbaegis | imho | 17:30 |
| sdake | 30 rock was good all the wy to the end | 17:30 |
| sdake | for those into more lighthearted idiot tube watching:) | 17:30 |
| Pavo | waiting for new Walking Dead | 17:31 |
| sdake | i tried to watch breakingbad | 17:31 |
| Pavo | breaking bad was awesome | 17:31 |
| sdake | but after the 4th episode, too intense for me ;) | 17:31 |
| kbaegis | sdake That's a great show. It keeps jumping the shark a little though | 17:32 |
| sdake | kbaegis a bit like the US election :) | 17:33 |
| Pavo | wow lol | 17:33 |
| kbaegis | sdake I'm trying to figure out if Trump or Clinton is the shark in this analogy :) | 17:33 |
| inc0 | speaking of which, house of cards | 17:33 |
| sdake | house of cards rocks | 17:34 |
| sdake | best scenee is the typewriter scene | 17:34 |
| Pavo | they both are bad and to think one of them is gonna be my commander in cheif | 17:34 |
| Pavo | so so sad | 17:34 |
| inc0 | I think both of them...and keep in mind that fish aren't actually famous by their intellectual brilliance | 17:34 |
| inc0 | for* | 17:34 |
| sdake | inc0++ | 17:35 |
| *** HyperJohnGraham has quit IRC | 17:35 | |
| kbaegis | inc0 lol | 17:35 |
| sdake | inc0 learning to mindread ;) | 17:35 |
| kbaegis | So I finally broke down and installed CentOS7 sdake | 17:37 |
| kbaegis | Since evidently no one likes supporting linux 4.8-rc2 :) | 17:37 |
| Pavo | nothing wrong with CentOS | 17:37 |
| Pavo | lol | 17:37 |
| kbaegis | Yeah, I'm the crazy guy who actually got kolla running w/ gentoo, remember | 17:38 |
| kbaegis | It works, but there are tons of workarounds | 17:38 |
| Pavo | I would imagine sp | 17:38 |
| Pavo | so* | 17:38 |
| kbaegis | Great learning opportunity :) | 17:39 |
| sdake | kbaegis glad you appreciated it;) | 17:39 |
| Pavo | is sad that were I work we using only Windows machines but the actual job requires me to know everything about Linux | 17:40 |
| Pavo | how messed up is that | 17:40 |
| sould | i feel your pain | 17:41 |
| inc0 | Pavo, such is life in big corpo.... | 17:42 |
| *** dwalsh has joined #openstack-kolla | 17:42 | |
| Pavo | yeah | 17:42 |
| Pavo | can't wait to retire, so I can actually get a job that pays more for my skillset | 17:43 |
| Pavo | 3 more years ugg | 17:43 |
| sould | almost there :) | 17:46 |
| kbaegis | Windows, those things you stare at instead of getting work done :) | 17:46 |
| Pavo | ^ +1000 | 17:46 |
| sdake | inc0 feels your pain too :) | 17:46 |
| sdake | at cisco we get a choice between mac or thinkpad | 17:46 |
| kbaegis | sdake You're at Cisco? | 17:47 |
| sdake | yup | 17:47 |
| Pavo | now that would awesome | 17:47 |
| kbaegis | sdake Cool. I was in AS for 5 years | 17:47 |
| sdake | mac gets no support | 17:47 |
| Pavo | nice I am military | 17:47 |
| sdake | kbaegis nice | 17:47 |
| Pavo | I love macs | 17:47 |
| sdake | pavo thank you for your service :) | 17:47 |
| Pavo | Ty | 17:47 |
| kbaegis | sdake All the hackers at Cisco get macs :) | 17:47 |
| Pavo | sdake CCIE? | 17:48 |
| kbaegis | Viso was the only reason I considered having a win machine | 17:48 |
| sdake | pavo not a big fan of certifications | 17:48 |
| Pavo | no reason for Visio anymore, Lucid or Giffy to the rescue | 17:48 |
| sdake | gliffy rocks | 17:49 |
| sdake | have to be a seroius rocket scientisst to get a ccie | 17:49 |
| sdake | sbezverk and daneyon have one | 17:49 |
| Pavo | lol yeah heard that exam is a nightmare | 17:49 |
| inc0 | Pavo, you'll gonna run kolla on tanks? damn our job just became even more awesome | 17:49 |
| sdake | pavo one piece of software i was responsible for birthing was corosync - used all over the military | 17:50 |
| Pavo | nah no longer combat arms, was Inantry for 12yrs then got my right knee blown off in Iraq back in 2007 then went 25B basically IT and now Cyber Security | 17:50 |
| Pavo | infantry* | 17:50 |
| Pavo | damn I can type | 17:51 |
| Pavo | nice sdake | 17:51 |
| *** huikang has quit IRC | 17:54 | |
| *** lyncos has quit IRC | 17:55 | |
| *** HyperJohnGraham has joined #openstack-kolla | 17:58 | |
| *** inc0 has quit IRC | 17:58 | |
| *** ad_rien_ has joined #openstack-kolla | 17:59 | |
| *** inc0 has joined #openstack-kolla | 17:59 | |
| *** lyncos has joined #openstack-kolla | 18:07 | |
| kfox1111 | sbezverk: back. | 18:14 |
| kfox1111 | looking at your email. | 18:14 |
| kfox1111 | sbezverk: ah. I think i see the issue. | 18:19 |
| kfox1111 | the annotation is in the wrong place. | 18:19 |
| kfox1111 | it needs to be in the spec->template->metadata | 18:19 |
| kfox1111 | here's a little snippet I've been using for validation tests: http://pastebin.com/zahuBbTd | 18:22 |
| kfox1111 | as its json encoded in yaml, sometimes you can mess up the format of the json and not know it. This helps debug it a little. | 18:22 |
| kfox1111 | hopefully they make it proper yaml soon. | 18:23 |
| Pavo | so.. what are the neutron mapping kolla uses? ie physnet0:br-ex? | 18:24 |
| Pavo | trying to add a network and need it to be flat for external network | 18:25 |
| inc0 | Pavo, default is physnet:<<neutron external interface >> | 18:25 |
| Pavo | ok | 18:25 |
| inc0 | https://github.com/openstack/kolla/blob/master/ansible/roles/neutron/templates/ml2_conf.ini.j2#L54 | 18:26 |
| Pavo | getting Failed to create nework | 18:26 |
| inc0 | actually it's physnet1 | 18:26 |
| inc0 | https://github.com/openstack/kolla/blob/ba30579b3b3e69536f5bd59e57d39ed4b63a2dd6/ansible/roles/neutron/defaults/main.yml#L61 | 18:26 |
| inc0 | so br-ex | 18:26 |
| inc0 | and br-ex is set up on neutron external interface | 18:27 |
| *** britthou_ is now known as britthouser | 18:28 | |
| Pavo | the /var/lib/docker/volumes/kolla_logs/_data/neutron/neutron-openvswitch-agent.log shows this http://pastebin.com/69r8dbNF | 18:30 |
| Pavo | looks like it messed up | 18:30 |
| *** dwalsh has quit IRC | 18:31 | |
| Pavo | nvm fixed it | 18:32 |
| *** HyperJohnGraham has quit IRC | 18:35 | |
| *** HyperJohnGraham_ has joined #openstack-kolla | 18:35 | |
| *** HyperJohnGraham_ is now known as HyperJohnGraham | 18:35 | |
| *** daneyon has joined #openstack-kolla | 18:36 | |
| openstackgerrit | Michal Jastrzebski (inc0) proposed openstack/kolla: Customization for base - centos https://review.openstack.org/358042 | 18:38 |
| *** daneyon has quit IRC | 18:41 | |
| *** lyncos has quit IRC | 18:48 | |
| *** egonzalez90 has joined #openstack-kolla | 18:59 | |
| pbourke | inc0: can you help us update init-runonce to create the correct networks for osic? | 19:00 |
| *** HyperJohnGraham has quit IRC | 19:02 | |
| inc0 | yeah | 19:02 |
| inc0 | let me get to tmux | 19:02 |
| *** ad_rien_ has quit IRC | 19:08 | |
| inc0 | pbourke, I think we need to restart .26 | 19:09 |
| inc0 | oh... | 19:10 |
| inc0 | pbourke, lets mount /dev/mapper/lxc-nova00 to /var/lib/docker | 19:10 |
| *** salv-orl_ has joined #openstack-kolla | 19:15 | |
| pbourke | inc0: could do, i really want to get going on these tests though | 19:15 |
| pbourke | inc0: seeing this hanging issue today | 19:15 |
| pbourke | inc0: ansible just sits there | 19:15 |
| *** salv-orlando has quit IRC | 19:17 | |
| inc0 | seems like this node is out of order on it's own | 19:17 |
| *** gfidente has quit IRC | 19:18 | |
| pbourke | which one | 19:18 |
| *** egonzalez90 has quit IRC | 19:18 | |
| inc0 | pbourke, ok, we should be good now | 19:22 |
| kfox1111 | sbezverk: looks like rdo does it with rootwrap. | 19:22 |
| *** salv-orl_ has quit IRC | 19:26 | |
| *** salv-orlando has joined #openstack-kolla | 19:27 | |
| sdake | gotta go - dentist appointment | 19:28 |
| inc0 | sdake, dont tell it's root canal again | 19:28 |
| sdake | cleaning i hope | 19:28 |
| sdake | definately not a root canal | 19:28 |
| sdake | unless they find something ominous | 19:29 |
| sdake | then that will be a different day | 19:29 |
| sdake | hey folks - welcome egonzalez90 to the CR team! | 19:29 |
| sdake | even though he just logged off for the day ;-) | 19:29 |
| sdake | if he needs any help getting started, please provide him a primer | 19:30 |
| sbezverk | kfox1111: strange we use the same image, it should aloso use rootwrap, all rootwrap configs are there?!?! | 19:31 |
| pbourke | inc0: why are we building everything again? | 19:32 |
| inc0 | i did git reset master | 19:33 |
| pbourke | damn someones been ignoring the gates | 19:35 |
| kfox1111 | sbezverk: guessing its just the ipc namespace thing then. | 19:36 |
| *** dwalsh has joined #openstack-kolla | 19:38 | |
| *** sdake has quit IRC | 19:41 | |
| Mech422 | So how's OSIC going ? | 19:51 |
| inc0 | going as we speak:) | 19:54 |
| inc0 | neutron is deploying now | 19:54 |
| Mech422 | inc0: oh cool - that should be a fun cluster to play with :-) | 19:54 |
| pbourke | inc0: fyi the rally stuff is ready to go | 19:54 |
| inc0 | cool! | 19:54 |
| pbourke | so the current deploy is scenario 1 again | 19:54 |
| Mech422 | I've got another week or two on the CloudStack stuff... then I should be back to Kolla | 19:54 |
| pbourke | should be quick enough to jump through them | 19:55 |
| inc0 | Mech422, I think it's fair to say that we all learned a lot during this excersises | 19:55 |
| Mech422 | inc0: oh? Any big take-away's ? | 19:55 |
| inc0 | besides that kolla is awesome? | 19:55 |
| Mech422 | LOL | 19:55 |
| inc0 | it took us a while to brush up our configs so we can run rally | 19:59 |
| inc0 | but we managed to deploy openstack probably over a dozen times in last 2 weeks | 19:59 |
| inc0 | big takeaway - elastic will eat through your storage space like there is no tomorrow | 20:00 |
| inc0 | I wonder if we could back it up on ceph | 20:00 |
| pbourke | inc0: I would say we need to limit it somehow | 20:01 |
| Mech422 | inc0: I don't see why not? You just just attach and RBD device to the host, mount/format it, and hand it to docker ? | 20:02 |
| inc0 | yeah, but as part of kolla deployment | 20:02 |
| pbourke | inc0: speaking of storage, we have users who dont like the data volumes thing as it lumps all data under /var/lib/kolla. Any appetite for making those configurable? | 20:02 |
| pbourke | s/kolla/docker | 20:02 |
| inc0 | well, that depends on docker really | 20:03 |
| inc0 | if docker can configure it, we do | 20:03 |
| rhallisey | pbourke, that's a docker change | 20:03 |
| inc0 | you can configure /var/lib/docker location tho | 20:03 |
| rhallisey | you can specifcy where you want your data | 20:03 |
| inc0 | which wiould be volumes + images | 20:03 |
| pbourke | its more the abstraction of the data | 20:03 |
| pbourke | people want to get logs under /var/log/kolla, not /var/lib/docker/volumes/_data/blah/blah | 20:04 |
| pbourke | same with /var/lib/nova and the like | 20:04 |
| pbourke | so I was thinking of making the list of volumes for the main images variables | 20:05 |
| pbourke | so they're configurable | 20:06 |
| inc0 | well named volumes are something bound to docker | 20:07 |
| inc0 | we would need to have different logic | 20:07 |
| inc0 | that being said ln -s /var/lib/docker/volumes/kolla_logs/_data /var/log/kolla works | 20:08 |
| pbourke | hmm | 20:12 |
| pbourke | i like it! | 20:12 |
| pbourke | will see if that does the trick, thanks inc0 | 20:12 |
| pbourke | rally tests are off now | 20:12 |
| pbourke | seem to take a while | 20:12 |
| inc0 | yeah | 20:13 |
| pbourke | if they all work, then dump json+html, then deploy next scenario | 20:13 |
| pbourke | britthouser: ^ | 20:13 |
| *** salv-orlando has quit IRC | 20:16 | |
| *** rhallisey has quit IRC | 20:23 | |
| *** daneyon has joined #openstack-kolla | 20:25 | |
| *** daneyon has quit IRC | 20:29 | |
| *** Jeffrey4l_ has joined #openstack-kolla | 20:34 | |
| *** Jeffrey4l has quit IRC | 20:35 | |
| *** dave-mccowan has joined #openstack-kolla | 20:51 | |
| *** ad_rien_ has joined #openstack-kolla | 20:55 | |
| *** sould has quit IRC | 20:55 | |
| *** kolladude has quit IRC | 20:57 | |
| *** fragatin_ has joined #openstack-kolla | 21:00 | |
| *** _coolsvap_ has quit IRC | 21:01 | |
| inc0 | it's done, next one with ceph | 21:02 |
| *** mark-casey has quit IRC | 21:03 | |
| *** fragatina has quit IRC | 21:04 | |
| *** jtriley has quit IRC | 21:16 | |
| *** tummy has joined #openstack-kolla | 21:18 | |
| *** haplo37__ has quit IRC | 21:19 | |
| *** ntpttr___ has quit IRC | 21:20 | |
| *** ntpttr___ has joined #openstack-kolla | 21:24 | |
| *** pbourke has quit IRC | 21:25 | |
| *** unicell has quit IRC | 21:28 | |
| *** unicell has joined #openstack-kolla | 21:28 | |
| *** michauds has quit IRC | 21:30 | |
| *** dave-mccowan has quit IRC | 21:33 | |
| *** sdake has joined #openstack-kolla | 21:35 | |
| *** sdake has quit IRC | 21:37 | |
| *** sdake has joined #openstack-kolla | 21:38 | |
| *** fragatin_ has quit IRC | 21:39 | |
| sdake | yo | 21:39 |
| *** fragatina has joined #openstack-kolla | 21:39 | |
| sdake | recruiters in a feeding frenzy on csco | 21:41 |
| inc0 | cisco has mass layoff or mass recriutment?> | 21:41 |
| sdake | i've got 8 recruitment spams today | 21:43 |
| sdake | typically on avg 1-2 a day | 21:43 |
| inc0 | I had something like that when intel announced big layoff | 21:43 |
| *** diogogmt has quit IRC | 21:48 | |
| Mech422 | inc0: cisco appears to be letting go a bunch of hardware types and hiring a bunch of software types | 21:53 |
| sdake | I currently have 4 Senior Software Engineer positions with my client eBay based in San Jose. These are full-time positions and they will accept visa holders. The salary is in the 150-180K range with some flexibility. | 21:55 |
| *** aNupoisc has joined #openstack-kolla | 21:55 | |
| sdake | eeeeee-bay | 21:55 |
| sdake | you know they are desparate when they include salary ranges in a recruiting email :) | 21:56 |
| aNupoisc | Hi guys I am trying to install ceph with kolla on a single node, so my OSD, monitor both will sit in the same machine | 21:56 |
| *** dmueller-evoila has quit IRC | 21:56 | |
| aNupoisc | I am using all-in-one inventory file with hostname as localhost but it fails saying no hosts found | 21:56 |
| aNupoisc | is that not the correct way to do | 21:56 |
| inc0 | aNupoisc, https://github.com/openstack/kolla/blob/master/doc/ceph-guide.rst#managing-ceph make sure to read this | 21:56 |
| aNupoisc | ? | 21:56 |
| aNupoisc | inc0: so that means it is a requirement to have 3 hosts atleast and one block device | 21:57 |
| aNupoisc | ? | 21:57 |
| inc0 | for ceph to work properly, yes, but there is hack to make it work aio | 21:58 |
| inc0 | however that's not problem you're having | 21:58 |
| inc0 | did you use default inventory or made any changes to it? | 21:58 |
| aNupoisc | inc0: i did followed this document and followed each step | 21:58 |
| sdake | kfox1111 about | 21:59 |
| aNupoisc | I use default inventory file | 21:59 |
| aNupoisc | inc0: i use default inventory file as there is nothing else than ansible_local and hostname as local in all blocks | 21:59 |
| aNupoisc | *localhost | 21:59 |
| inc0 | paste plz your ansible output | 21:59 |
| aNupoisc | inc0: give me just few minute, it is setup on my other machine | 22:00 |
| *** salv-orlando has joined #openstack-kolla | 22:00 | |
| sdake | i'm not getting paid enough ;) | 22:00 |
| inc0 | sdake, lol | 22:01 |
| inc0 | I wish I'd be paid as much as you are | 22:01 |
| sdake | If you are not interested, I am almost positive you know of someone who might be interested and a good fit. For that, I will pay out a $1200 bonus if we can close this position as a result of your referral. Please introduce me to the referred person by giving me their name & email or phone. | 22:04 |
| sdake | anyone wnat a job at ebay? 1200 bucks would be fun ;) | 22:04 |
| sdake | where did the kevin fox go | 22:07 |
| aNupoisc | inc0: here is my globals.yml http://pastebin.com/w74wLDg6 and here is my ansible output http://pastebin.com/XhQsHdT1 | 22:07 |
| aNupoisc | i am now getting host unreachable | 22:07 |
| aNupoisc | as i changed the kolla_internal_ip_address to my current machine | 22:08 |
| sdake | aNupoisc you shoudln't do that imo | 22:08 |
| sdake | that will crater your network | 22:08 |
| aNupoisc | is it? I thought as kolla always uses current machine's address for internal_vip_address | 22:09 |
| inc0 | aNupoisc, btw Anup?;) btw Michal here | 22:09 |
| aNupoisc | Yup its Anup Michal | 22:09 |
| sdake | Join us at the Microsoft | 22:09 |
| aNupoisc | oh | 22:09 |
| inc0 | sdake, really? | 22:09 |
| sdake | boy you got to be kiddding | 22:09 |
| sdake | do these poeple even read my linked in profile | 22:09 |
| inc0 | well, they did recruit kubernetes lead engineer like a month ago | 22:10 |
| sdake | good point | 22:10 |
| inc0 | so there is something interesting out there | 22:10 |
| aNupoisc | sdake: what is there at Microsoft? | 22:10 |
| aNupoisc | some conference? | 22:10 |
| sdake | a job apparently | 22:10 |
| aNupoisc | sdake: haha | 22:11 |
| sdake | microsoft turne dme down out of college | 22:11 |
| inc0 | pain never healed fully? | 22:11 |
| sdake | nope | 22:11 |
| inc0 | Mirantis turned me down, same thing | 22:11 |
| sdake | i had ot suck it up at intel instead out of university | 22:12 |
| sdake | a hardware company | 22:12 |
| sdake | for a software nerd | 22:12 |
| sdake | int he 90s | 22:12 |
| sdake | not ideal | 22:12 |
| sdake | during the dot com crash | 22:12 |
| aNupoisc | sdake: inc0: there is no pain at least for now | 22:12 |
| aNupoisc | haha | 22:12 |
| ntpttr___ | inc0: this conversation led me to your linkedin | 22:13 |
| ntpttr___ | inc0: you look a lot like eligah wood without your beard | 22:13 |
| ntpttr___ | elijah* | 22:13 |
| sdake | inc0 has a pic without a beard? | 22:13 |
| ntpttr___ | https://www.linkedin.com/in/inc007?authType=name&authToken=_KpT&trk=prof-connections-name | 22:13 |
| inc0 | it's an lod photo | 22:13 |
| inc0 | old | 22:13 |
| sdake | ya i remember hwen michal linked in with me | 22:13 |
| sdake | and i had no idea who he was | 22:13 |
| sdake | but whatever ;) | 22:14 |
| ntpttr___ | that's a young frodo baggins if I've ever seen one | 22:14 |
| aNupoisc | ntpttr___ sdake: inc): curious to see everyone's profile now | 22:14 |
| aNupoisc | hehe | 22:14 |
| sdake | michl we have known each other for about a year :) | 22:14 |
| aNupoisc | ntpttr___ why have you put a looong underscore after your name. ;) | 22:14 |
| ntpttr___ | aNupoisc: My IRC bouncer is having technical difficulties and I can't access it atm... | 22:15 |
| ntpttr___ | I need to fix it at home | 22:15 |
| ntpttr___ | so it's logged in but I can't reach it basically | 22:15 |
| aNupoisc | lol | 22:15 |
| aNupoisc | okay | 22:15 |
| inc0 | sdake, close to 1.5yrs now | 22:16 |
| aNupoisc | sdake: inc0: so does kolla_internal_vip_address you are assuming might be wrong? | 22:16 |
| inc0 | but this photo is like 6yrs old | 22:16 |
| sdake | aNupoisc interna lvip address should be unused address on your network | 22:16 |
| inc0 | aNupoisc, something much more wrong is happening;) node unreachable == try to ssh to it manually | 22:16 |
| inc0 | sdake, but tbh, your photo is not exactly actual as well now isn't it?;) | 22:17 |
| aNupoisc | inc0: sdake: ah okay, I am actually not using --private-key while running deploy command | 22:17 |
| aNupoisc | so ssh login is expected to fail | 22:18 |
| sdake | inc0 its a head shot | 22:18 |
| sdake | inc0 ever done those before? | 22:18 |
| inc0 | only in computer games | 22:18 |
| aNupoisc | let me try to use private key and then ssh login | 22:18 |
| *** tummy has quit IRC | 22:18 | |
| sdake | some day you will need one ;) | 22:18 |
| *** dmueller-evoila has joined #openstack-kolla | 22:18 | |
| sdake | probably 33-36 range | 22:18 |
| sdake | or you could get one now u pto you | 22:18 |
| inc0 | yeah, when Intel will feel compelled to show me around, but for now I'm just lowly footslogger, no need for fancy photos | 22:19 |
| sdake | inc0 bro ure almost on a world stage already - you just don't know it ;) | 22:19 |
| openstackgerrit | Merged openstack/kolla-kubernetes: Adding NOTE for Kubernetes Secret Generator https://review.openstack.org/356719 | 22:20 |
| inc0 | sshhhh not so loud or recruiters from microsoft will hear | 22:20 |
| *** JoseMello has quit IRC | 22:20 | |
| inc0 | anyway I'm going for corporate dinner (yay for free food), cya around | 22:21 |
| inc0 | sdake, if you have a while, there is deployment going on in osic cluster | 22:21 |
| sdake | inc0 i'e had a rough week | 22:21 |
| inc0 | when it finishes, would you plz run rally and dump results to ~/rally-scenerios/<< the later dir >> | 22:21 |
| sdake | followed by dentist | 22:21 |
| inc0 | fair enough | 22:21 |
| sdake | so prettym much no | 22:21 |
| inc0 | ok, I'm off | 22:23 |
| inc0 | have good weekend folks | 22:23 |
| *** dmueller-evoila has quit IRC | 22:25 | |
| *** tummy has joined #openstack-kolla | 22:29 | |
| Pavo | so... after finally getting running setup with 2 nodes, separate controller, compute. Tried adding all the services to both nodes for more of a HA fashion and gets stuck at starting neutron_server container on second node lol | 22:33 |
| sdake | pavo this is bare metal ? | 22:33 |
| Pavo | nah, same VMs | 22:34 |
| sdake | you sure your interfaces are the same ordering on each vm? | 22:34 |
| sdake | that sounds alot like you just cratered your network because you hve your interfaces backwards | 22:34 |
| Pavo | yes, used same globals just changed inventory | 22:35 |
| sdake | but what about the vms? | 22:35 |
| Pavo | all the same as before | 22:35 |
| aNupoisc | inc0: hey so i was able to ssh to the machine using the key | 22:36 |
| Pavo | its all good, gonna clean the nodes and try again | 22:36 |
| Pavo | then gonna test adding a 3rd node | 22:36 |
| aNupoisc | inc0: and now I have again changed the kolla_internal_vip_address to something unused | 22:36 |
| inc0 | yeah | 22:37 |
| aNupoisc | but still the ansible fails | 22:37 |
| aNupoisc | throwing unreachable | 22:37 |
| inc0 | ansible can't ssh for some reason | 22:37 |
| inc0 | you need to debug this | 22:38 |
| inc0 | unfortunately I gtg | 22:38 |
| aNupoisc | actually this is strange I was first getting "no hosts found error" and now unreachable | 22:38 |
| aNupoisc | but then do i need to change all-in-one? | 22:38 |
| inc0 | talk to you guys later | 22:38 |
| inc0 | ! | 22:38 |
| aNupoisc | Okay. Happy friday | 22:38 |
| inc0 | aNupoisc, you shouldn't | 22:38 |
| aNupoisc | okay cool | 22:38 |
| inc0 | something is wrong in your env, that's my guess | 22:38 |
| inc0 | anyway, see you all later | 22:38 |
| Mech422 | aNupoisc: are you using IP address or hostname | 22:39 |
| Mech422 | aNupoisc: if hostname, does it resolv properly | 22:39 |
| *** inc0 has quit IRC | 22:39 | |
| aNupoisc | Mech422: IP address or hostname in what? | 22:39 |
| Mech422 | aNupoisc: inventory file | 22:39 |
| Mech422 | ssh can't find/contact your host | 22:39 |
| aNupoisc | Mech422: oh in inventory I am using IP address | 22:40 |
| Mech422 | ok | 22:40 |
| aNupoisc | let me paste that | 22:40 |
| aNupoisc | I have actually altered it | 22:40 |
| Mech422 | aNupoisc: and you setup ansible to use the correct user for ssh ? | 22:41 |
| aNupoisc | Mech422: here is my inventory http://pastebin.com/38DJy4T0 | 22:41 |
| sdake | aNupoisc there is only one host in that inventory file | 22:42 |
| Mech422 | seems like your trying to do an all-in-one install to a remote host ? | 22:42 |
| aNupoisc | sdake: yes I am doing it on that machine | 22:42 |
| aNupoisc | sdake: MEch422: let me change it tp localhost again | 22:43 |
| aNupoisc | sdake: Mech422: http://pastebin.com/UUrzfx2G | 22:46 |
| aNupoisc | this is the new file | 22:46 |
| sdake | aNupoisc localhost=10.3.66.64? | 22:46 |
| aNupoisc | Yes. | 22:46 |
| sdake | hve you palced that in your /etc/host file? | 22:46 |
| aNupoisc | sdake: it is my current machine where I want to setup ceph | 22:46 |
| sdake | before setting u pceph | 22:47 |
| sdake | get a working multinode deployment imo | 22:47 |
| *** ad_rien_ has quit IRC | 22:47 | |
| Pavo | sdake here is my globals, http://pastebin.com/VekDu45P and here is my multinode inventory http://pastebin.com/01PbdfxD. Anything look wrong? | 22:48 |
| sdake | line 26 is wrong | 22:48 |
| Pavo | in? | 22:48 |
| sdake | or altnerivately your host ip is wrong | 22:48 |
| sdake | in that goba.syml | 22:48 |
| aNupoisc | sdake: in my /etc/hosts i have 127.0.0.1 as localhost 10.3.66.64 as otc-ironic(myhostname) | 22:49 |
| Pavo | kolla_internal_vip_address: "192.168.1.254" | 22:49 |
| sdake | ya - that is not on the same network as 10.3.66.64 | 22:49 |
| Pavo | I don't have a network of 10.3.66.64 | 22:49 |
| *** tummy has quit IRC | 22:49 | |
| sdake | anything in inventory file should be a mangement network ip | 22:49 |
| aNupoisc | Mech422: 127.0.0.1 localhost and 10.3.66.64 otc_ironic(hostname) | 22:49 |
| sdake | you do in your previous inventory file | 22:50 |
| aNupoisc | sdake: Mech422: should the /etc/hosts/ should have localhost 10.3.66.64 | 22:50 |
| sdake | nope | 22:50 |
| *** dwalsh has quit IRC | 22:50 | |
| Pavo | think you might be thinking of someone else lol, I have always had only 2 networks, 192.168.1.0/24 and 192.168.188.0/24 | 22:50 |
| sdake | it should have a machine name insted of localhost | 22:50 |
| sdake | pavo sorry got mixed up between you and aNupoisc | 22:51 |
| Pavo | yeah figured that, your brain is getting fried brother | 22:51 |
| sdake | aNupoisc paste your globals.yml please | 22:51 |
| aNupoisc | sdake: haha yeah i was reading how is his ip address same as mine | 22:51 |
| aNupoisc | :) | 22:51 |
| sdake | pavo tell me about it | 22:51 |
| sdake | i hate the dentist | 22:51 |
| sdake | with a pasion | 22:51 |
| Pavo | me too | 22:51 |
| Pavo | I have to have oral surgery next month :( | 22:52 |
| sdake | my condolences | 22:52 |
| Pavo | so.... sdake does anyting in my globals and inventory look wrong for having all services running on both nodes in a HA fashion? | 22:53 |
| sdake | pavo what is the result of doing a deploy? | 22:54 |
| sdake | crater on openvswitch start? | 22:54 |
| sdake | or neutron start? | 22:54 |
| Pavo | stuck on neutron_server container on second node, first node gets done fine | 22:54 |
| sdake | pavo your interfaces are probably mixed around on the second host | 22:55 |
| aNupoisc | sdake: Mech422: here is my globals.yml http://pastebin.com/WjrPrvcY and my inventory file http://pastebin.com/UUrzfx2G and my /etc/hosts http://pastebin.com/8x9AXhjz | 22:55 |
| sdake | try switching the network cables on the back of the gear | 22:55 |
| Pavo | roger | 22:55 |
| sdake | on the 2nd node pavo not on the first | 22:55 |
| Pavo | roger | 22:55 |
| aNupoisc | sdake: are you having some dental surgery? | 22:55 |
| sdake | aNupoisc no | 22:55 |
| aNupoisc | sdake: Pavo: I am scared of dentist | 22:56 |
| sdake | aNupoisc just a cleaning where the flossing broke TWO of my fillings | 22:56 |
| * sdake growls | 22:56 | |
| sdake | bad bad mood | 22:56 |
| Pavo | aNupoisc me too, everytime I got something bad or worse happens | 22:56 |
| aNupoisc | sdake: ahh and that too on Fri-Yay ;) | 22:56 |
| sdake | ya dentist is like "nah there wont be any sensitivity" | 22:57 |
| sdake | bullshit | 22:57 |
| sdake | hurts like a mofo | 22:57 |
| aNupoisc | sdake: seriously they always say such crap | 22:57 |
| sbezverk | sdake: you need to ask extra shot of something.. ;-) | 22:57 |
| sdake | shot of morphine in the ass maybe | 22:58 |
| Mech422 | sdake: I had like 12 teeth pulled in the last couple of months | 22:58 |
| sdake | ok sbezverk on to your issue | 22:58 |
| sdake | paste pl | 22:58 |
| sdake | Mech422 yieks! | 22:58 |
| Mech422 | sdake: finally getting my partial lower done | 22:58 |
| sdake | i hae all original teeth | 22:58 |
| sdake | but lots o crowns | 22:58 |
| sdake | if I lose a tooth permanently -> implants | 22:58 |
| Pavo | one dentist once told me that they actually use the metal stuff on purpose because it causes pain, so people will learn to have better hygiene, I was thats so..... messed up | 22:58 |
| Mech422 | sdake: I got kicked in the mouth as a kid - my front lower and jaw are all messed up | 22:59 |
| sdake | Mech422 sorry to hear it | 22:59 |
| Mech422 | eh - it was nothing compared to all my wisdom teeth coming in wrong | 22:59 |
| sdake | ya i had those pulled at 16 | 22:59 |
| Pavo | yeah I had all the enamel broken off on mine of being blown up 4 times in Iraq, so they are just dying | 22:59 |
| Mech422 | I'm actually glad to be rid of them - nothing hurts for the first time in like 10-15 years | 23:00 |
| *** salv-orlando has quit IRC | 23:00 | |
| sdake | pavo war is hell :( | 23:00 |
| *** v1k0d3n has quit IRC | 23:01 | |
| Pavo | meh | 23:01 |
| Pavo | been deployed 6 times, 4 times to Iraq, 2 times to Afghan | 23:01 |
| *** v1k0d3n has joined #openstack-kolla | 23:01 | |
| Pavo | it is what it is | 23:01 |
| Mech422 | Pavo: wow - I didn't even know that was possible | 23:01 |
| sdake | wow - pretty hard core | 23:01 |
| aNupoisc | Mech422: sdake: wisdom teeth surgery is baaaad | 23:01 |
| Pavo | 67 months total deployment time | 23:01 |
| *** v1k0d3n has quit IRC | 23:01 | |
| sdake | they wouldn't let me in the military | 23:01 |
| sdake | even if i wanted in | 23:02 |
| aNupoisc | Pavo: you were in military? | 23:02 |
| Pavo | yeah Mech422 the force of an explosion is pretty bad on your teeth | 23:02 |
| Mech422 | aNupoisc: I didn't have the surgery ... But when they come in and break/crack your other teeth, its a bitch | 23:02 |
| *** v1k0d3n has joined #openstack-kolla | 23:02 | |
| Pavo | I am in the military | 23:02 |
| aNupoisc | Pavo: Woww | 23:02 |
| sdake | i have a good riend who was in the rangers | 23:02 |
| Mech422 | Pavo: of all the things I would think it would be bad for....I'd have never guessed that | 23:03 |
| sdake | he said "bigger the gob better the job" is the dentist motto in the military ;) | 23:03 |
| Pavo | it wasn't so bad, only main one that really sucked was when I got my right knee blown off with a EFP | 23:03 |
| aNupoisc | Mech422: Oh thank god still my wisdom teeth are giving that pain | 23:03 |
| Pavo | lol sdake | 23:03 |
| sdake | his kneeds afu because of jumping out of helicopters and airlaines and whatnot | 23:03 |
| sdake | pavo ya that sounds rough | 23:04 |
| aNupoisc | haha sdake | 23:04 |
| sdake | sbezverk can you pasate here plz | 23:04 |
| aNupoisc | sdake: did you got chance to see my globals.yml | 23:04 |
| aNupoisc | and other files | 23:04 |
| sdake | sbezverk my next suggestion was going to be -> look in v/ar/run/openvswitchd in /run | 23:04 |
| aNupoisc | Mech422: | 23:05 |
| sdake | aNupoisc one person at a tiem guys- i'm not feeling super | 23:05 |
| aNupoisc | sdake: yes actually | 23:05 |
| aNupoisc | i will wait | 23:05 |
| *** v1k0d3n has quit IRC | 23:05 | |
| *** v1k0d3n has joined #openstack-kolla | 23:05 | |
| sdake | speaking of v1k0d3n | 23:05 |
| Mech422 | aNupoisc: what happens when you try with the current setup ? | 23:05 |
| sdake | wtb | 23:05 |
| *** daneyon has joined #openstack-kolla | 23:07 | |
| aNupoisc | Mech422: sdake: oh wow i am getting some new error http://pastebin.com/1ZKbRkLb | 23:07 |
| sdake | aNupoisc run kolla-deploy with sudo pls | 23:07 |
| Mech422 | aNupoisc: your ansible user doesn't have passwordless sudo setup ? | 23:08 |
| * jrich523 takes a number | 23:08 | |
| Mech422 | aNupoisc: its a perms error... | 23:08 |
| sbezverk | sdake: what do you want me to paste, lots track, sorry | 23:08 |
| sdake | ok folks - debugging 101 -> go from simple to complex | 23:08 |
| sdake | sbezverk the last docker inspect | 23:08 |
| Pavo | so... I think I might have figured what the issue is with the neutron server getting stuck during deployment | 23:09 |
| sdake | pavo cables swapped? | 23:09 |
| Pavo | damn vmware fusion keeps asking for my password when the interfaces go into promisc mode | 23:09 |
| Pavo | bah | 23:10 |
| sdake | pavo wierd | 23:10 |
| sbezverk | sdake: http://paste.openstack.org/show/561475/ | 23:10 |
| Pavo | yeah | 23:10 |
| aNupoisc | Mech422: sdake: yes there was no permission apparently and I forgot to run using sudo | 23:11 |
| sdake | aNupoisc kolla can be run without sudo but its harder to setup and undocumented | 23:11 |
| *** daneyon has quit IRC | 23:12 | |
| aNupoisc | sdake: yeah I have documented it though on my side coz sometimes the errors that come are pretty weird to get to know the actual reason of failure | 23:12 |
| *** peterjenkins has quit IRC | 23:13 | |
| *** smekel_ has quit IRC | 23:13 | |
| aNupoisc | sdake: Mech422: now it is looking for Docker module, it is unable to find that. | 23:14 |
| aNupoisc | may be i will install docker | 23:14 |
| Mech422 | aNupoisc: yeah - the hosts your deploying to need docker | 23:15 |
| sdake | aNupoisc precheck ftw :) | 23:15 |
| aNupoisc | sdake: Mech422: yeah i did not run prechecks | 23:15 |
| *** peterjenkins has joined #openstack-kolla | 23:15 | |
| aNupoisc | it is not mentioned in https://github.com/openstack/kolla/blob/master/doc/ceph-guide.rst#managing-ceph so missed | 23:16 |
| *** smekel_ has joined #openstack-kolla | 23:18 | |
| *** zhurong has joined #openstack-kolla | 23:18 | |
| aNupoisc | sdake: Mech422: let me run run prechecks.yml first that would be better | 23:19 |
| aNupoisc | sdake: Mech422: Is there any yml file that installs all the packages,services required for kolla setup? | 23:20 |
| aNupoisc | sdake: Mech422: i think the machine should have many packages like python-pip,python-dev,libssl,docker right? | 23:21 |
| Mech422 | aNupoisc: I don't think so - I have my own ansible plays that set everything up | 23:21 |
| Mech422 | aNupoisc: yeah - lots of stuff to install | 23:22 |
| sdake | there are playbooks in the review queue that setup the hosts | 23:22 |
| sdake | but its master only | 23:22 |
| sdake | and wont be backported | 23:22 |
| sdake | so that wont reallly help you | 23:22 |
| aNupoisc | sdake: Mech422: okay then let me do that otherwise the deploy ansible might fail at various sections | 23:23 |
| aNupoisc | Mech422: that's good man | 23:23 |
| aNupoisc | Mech422: even I will prepare something like taht | 23:23 |
| Pavo | would be nice to have a playbook to setup the target host | 23:24 |
| aNupoisc | Mech422: sdake: my very first concern was it was not recognizing the hosts and that got resolved. | 23:24 |
| Mech422 | aNupoisc, Pavo: here's a partial dump of my setup plays: http://paste.openstack.org/show/561487/ | 23:25 |
| Mech422 | it shows the kolla specific stuff I install | 23:26 |
| Mech422 | I also install other crap, so the list might not be exhaustive | 23:26 |
| Pavo | nice Mech422 but is that for latest Kolla or stable? | 23:26 |
| Mech422 | (like I install openvswitch before I install kolla, so its not listed ) | 23:26 |
| Mech422 | Pavo: master | 23:27 |
| Mech422 | Pavo: ansible 2.1 | 23:27 |
| Pavo | ah | 23:27 |
| Pavo | yeah I'm running stable | 23:27 |
| Mech422 | Pavo: just change the ansible=2.1.0.0 to ansible=1.9.x.y ? | 23:27 |
| *** dmueller-evoila has joined #openstack-kolla | 23:28 | |
| sean-k-mooney | the kolla host playbook i have in review is for master but it might work with stable. i could probaly make the ansible version setable | 23:28 |
| Pavo | here is all the cmds I used on my deployer, ie.. master | 23:28 |
| Pavo | http://pastebin.com/NSbEcZ1C | 23:28 |
| aNupoisc | Mech422: much appreciate man. But do we need to create virtual env? Or is that for your setup? | 23:28 |
| Pavo | and for all target nodes http://pastebin.com/q0bNJ5e1 | 23:29 |
| Mech422 | aNupoisc: thats just me | 23:29 |
| Mech422 | aNupoisc: ignore everything except the package names | 23:29 |
| aNupoisc | Mech422: okay and also we need to "mount --make-shared /run" | 23:29 |
| aNupoisc | is that right? | 23:30 |
| aNupoisc | Mech422: okay | 23:30 |
| Mech422 | Pavo: very nice :-) | 23:31 |
| sean-k-mooney | aNupoisc: if you are on ubuntu 14.04 yes | 23:31 |
| Mech422 | Pavo: looks like you have to install a lot less then me :-P | 23:31 |
| * Mech422 mutters about ubuntu | 23:31 | |
| aNupoisc | sean-k-mooney: yes i am using that | 23:31 |
| Pavo | I have never really liked ubuntu myself | 23:31 |
| Mech422 | I liked it a lot more before I started playing with Kolla | 23:32 |
| Mech422 | :-P | 23:32 |
| Mech422 | Not sure I'm gonna like the systemd crap in 16.04 | 23:32 |
| sean-k-mooney | after years of devstack where i had to support ubuntu centos and fedora ubuntu has less issues but theyare all a pain | 23:33 |
| Mech422 | it was a real pain to find docs on convertying my udev rules for renaming interfaces to systemd .link files | 23:33 |
| aNupoisc | Mech422: Pavo: sean-k-mooney: you can also add --insecure-registry option in /etc/default/docker in this setup file | 23:33 |
| Pavo | you can yes | 23:33 |
| Pavo | I use a secure registry though | 23:33 |
| sean-k-mooney | aNupoisc: yes my kolla host playbook will do that for you | 23:33 |
| Mech422 | aNupoisc: yeah - Pavo does it right, I'm lazy and use insecure | 23:33 |
| *** zhurong has quit IRC | 23:34 | |
| aNupoisc | here is my setup ansible http://pastebin.com/fUhusWqi | 23:34 |
| Pavo | I need to start getting into ansible | 23:35 |
| Mech422 | aNupoisc: looks good :-) | 23:35 |
| Pavo | has alot of power that I could leverage to automated slot of stuff | 23:35 |
| *** dmueller-evoila has quit IRC | 23:35 | |
| Mech422 | Pavo: oh hey - do you happen to know anything about generating certs for openvpn with easy-rsa? | 23:35 |
| aNupoisc | sean-k-mooney Mech422:I also use insecure ;) | 23:35 |
| sdake | sbezverk - check out this path on the HOST : "/var/run/openvswitch:/var/run/openvswitch", | 23:35 |
| sdake | see if it contains a socket file | 23:36 |
| aNupoisc | sean-k-mooney: can you paste your host playbook may be i missed it | 23:36 |
| sbezverk | sdake: it does | 23:37 |
| sean-k-mooney | aNupoisc: sure its availble for review here https://review.openstack.org/#/c/325631/8 | 23:37 |
| patchbot | sean-k-mooney: patch 325631 - kolla - add baremetal role to install kolla deps. | 23:37 |
| sbezverk | sdake: it seems it is in "good: phase now. I restarted multiple times and it holds.. | 23:37 |
| sean-k-mooney | aNupoisc: i will be adding the bootstrap-build-host part later next week | 23:38 |
| sdake | sbezverk so - your all set then? | 23:38 |
| sdake | sbezverk mind sharing with the channel what you learned about yaml files | 23:38 |
| sbezverk | sdake: for these two process at this point of time yes, but I have no confidence it will not break in next 10 minutes | 23:39 |
| Pavo | sdake does kolla supprt trunking network_interface and neutron_external_interface? | 23:39 |
| sdake | pavo if by trunking you mean bonding | 23:39 |
| sdake | pavo yes we have it setup that way in osic | 23:39 |
| Pavo | trunking I mean by 802.1q | 23:40 |
| Pavo | allowing multiple vlans | 23:40 |
| sean-k-mooney | Pavo: as in vlan trunking | 23:40 |
| sdake | pavo i am not a networking expert, but I believe we setup 802.1q mode in bonding driver | 23:40 |
| Pavo | roger | 23:40 |
| Pavo | thats good | 23:40 |
| sdake | no idea what vlan trunking is sorry :) | 23:40 |
| Pavo | ah | 23:40 |
| sdake | we setup some 80xxxx mode in bonding ddriver | 23:41 |
| sbezverk | for folks intersted in kube. if you use construct: command/args to execute a command in a pod, it is safer to provide arguments on each line instead of on one line separated by comma | 23:41 |
| sdake | i could be off on the actual mode | 23:41 |
| Pavo | I am a networking geek myself | 23:41 |
| sean-k-mooney | pavo you can check out the osic network config in the ether pad | 23:41 |
| aNupoisc | sean-k-mooney: great!! | 23:41 |
| sdake | sbezverk mind pasting a before and after for kfox1111 | 23:41 |
| Pavo | sean-k-mooney doesn't that require kubernates? | 23:41 |
| sean-k-mooney | Pavo: we were testing with both bonds and vlan subports | 23:41 |
| sbezverk | sdake: kfox1111 always using right way :-) | 23:41 |
| aNupoisc | also do we need configuring virtualization? I mean kvm everytime | 23:42 |
| sean-k-mooney | Pavo: doent what require kuberneties? | 23:42 |
| sdake | configure how? | 23:42 |
| sdake | insmod? | 23:42 |
| sdake | aNupoisc if you mean insmod, no kolla does that for you | 23:42 |
| Pavo | osic sean-k-mooney ? | 23:42 |
| openstackgerrit | Shaun Smekel proposed openstack/kolla: Add dockerfiles for keystone fernet https://review.openstack.org/351139 | 23:42 |
| sean-k-mooney | osic no | 23:43 |
| sdake | pavo open stack innovation center is providing us with 131 nodes for kolla scale testing | 23:43 |
| Pavo | ah lol | 23:43 |
| Pavo | I'm a newb | 23:43 |
| Pavo | mybad | 23:43 |
| sdake | initial results look good | 23:43 |
| sdake | we have documented the setup in an etherpad as well as a gerrit review | 23:43 |
| Pavo | nice | 23:43 |
| aNupoisc | sdake: i mean just enable virtualization feature on that machine | 23:43 |
| sdake | and eventually it will be published on docs.oo | 23:43 |
| sbezverk | sdake: sean-k-mooney: I have concern about location of ovs db. Do you mind share your thoughts about it. | 23:43 |
| *** dmueller-evoila has joined #openstack-kolla | 23:43 | |
| sdake | aNupoisc - you have to do that in bios | 23:43 |
| aNupoisc | for me its intel kvm module | 23:43 |
| Pavo | well I would really clean the docs up before adding something that complex, lol just saying | 23:44 |
| sean-k-mooney | testing kolla-k8s is out of scope currently but it would be interesting to do maybe next cycle when it gets closer to a complete solutio n | 23:44 |
| openstackgerrit | Shaun Smekel proposed openstack/kolla: Add dockerfiles for keystone fernet https://review.openstack.org/351139 | 23:44 |
| sdake | pavo - its a new doc - like to see? | 23:44 |
| Pavo | sure why not | 23:44 |
| Pavo | lol | 23:44 |
| sean-k-mooney | sbezverk: sure | 23:44 |
| aNupoisc | sdake: but no need to change anything in system files right? | 23:44 |
| sdake | pavo: https://review.openstack.org/#/c/352101/ | 23:45 |
| patchbot | sdake: patch 352101 - kolla - Add OSIC Scale Testing Documentation | 23:45 |
| sean-k-mooney | sbezverk: what are you worried about in terms of the ovsdb? | 23:45 |
| Mech422 | aNupoisc: actually, I think you need to enable tcp listener and stuff for live migration | 23:45 |
| sbezverk | sdake: sean-k-mooney: Originally ovs db was located on a host mount and I noticed that this approach is not ideal as from time to time I see some leftovers from previous runs.. | 23:45 |
| sdake | jrich523 your up | 23:45 |
| aNupoisc | sdake: i just remember sometime in between using "modprobe kvm-intel" so thought to ask | 23:45 |
| jrich523 | lol awesome thanks :) | 23:45 |
| aNupoisc | Mech422: okay then I should not worry | 23:45 |
| sdake | aNupoisc - we do that automatically | 23:45 |
| jrich523 | im trying to get the ansible compute deployment working | 23:45 |
| sbezverk | then I decided to try to use it on local to the pod volume, since no need other processes accessing directly | 23:46 |
| jrich523 | so i took the all-in-one and basically added my host in the [compute] section at the top like so: 1.2.3.4 ansible-connection=ssh | 23:46 |
| sbezverk | this approach also suffers from some issue. example if pod gets restarted that volume is gone and previous config needs to be replayed | 23:46 |
| sdake | jrich523 i'd start with the multinode example | 23:46 |
| jrich523 | which seemed cool? so i then attempted to import it like you said, kolla-ansible -i filename | 23:47 |
| jrich523 | ok | 23:47 |
| jrich523 | but, the real problem (i think) is the syntax for adding it | 23:47 |
| sdake | i think you dont want ansible-connection=ssh | 23:47 |
| sdake | but i am not sure | 23:47 |
| jrich523 | it basically just spews help | 23:47 |
| Pavo | ouch latency seems pretty high | 23:47 |
| sdake | kolla-ansible -i /etc/kolla/multinode-inventory deploy | 23:47 |
| jrich523 | ahh missed deploy | 23:48 |
| jrich523 | i'll take a look at multinode before i do this tho | 23:48 |
| jrich523 | thanks | 23:48 |
| sean-k-mooney | sbezverk: well when a new ovsdb container is spawned the ovsdb should be iniilised from a clean copy of the schema and bootstraped correct. | 23:48 |
| sean-k-mooney | sbezverk: is that the case with the k8s deployment | 23:48 |
| sdake | pavo ya those are old results | 23:48 |
| Pavo | ah | 23:48 |
| sdake | pavo we had a kernel problem in the scale testing | 23:48 |
| sbezverk | so my question is should stick with host location and try to deal with leftovers or keep it on local container space and see how old config can be replayed. | 23:48 |
| sdake | we are in process of rerunning the tests | 23:48 |
| Pavo | only 4 control nodes for 101 computes? | 23:48 |
| sdake | we have about 10-12 scenarios | 23:48 |
| Pavo | seems not enough | 23:48 |
| sdake | pavo our model is 3 control + 90 compute + 8 storage | 23:49 |
| sdake | or a ratio thereof | 23:49 |
| openstackgerrit | Shaun Smekel proposed openstack/kolla: Add full support for fernet https://review.openstack.org/349366 | 23:49 |
| sbezverk | sean-k-mooney: yes it is bot kube | 23:49 |
| Pavo | looks very promising and interesting | 23:49 |
| sbezverk | I mean for kube case | 23:49 |
| Pavo | would love to see a networking diagram | 23:49 |
| sdake | pavo however you could run 9 control node 90 compute if you like | 23:49 |
| sdake | 3 control nodes is plenty for 100 nodes | 23:49 |
| openstackgerrit | Shaun Smekel proposed openstack/kolla: Add full support for fernet https://review.openstack.org/349366 | 23:50 |
| sean-k-mooney | sdake 1 contoler should be plenty for 100 compute nodes but 3 for ha | 23:50 |
| sdake | sean-k-mooney agree | 23:50 |
| Pavo | seriously? | 23:51 |
| sean-k-mooney | sbezverk: i would lean towords keeping the ovsdb local to the container | 23:51 |
| Pavo | seems like it would be api overload lol | 23:51 |
| sean-k-mooney | Pavo: it depends on how beefy the contoler is | 23:51 |
| Pavo | true | 23:52 |
| Pavo | we plan on using a quad 40Gb DAC for our apis | 23:52 |
| *** dmueller-evoila has quit IRC | 23:52 | |
| sean-k-mooney | sbezverk: the ovs agent will recreate the flows and interconnect the bridges as needed when it connects to the ovsdb | 23:52 |
| sbezverk | sean-k-mooney: thanks I will see how replay might be done | 23:52 |
| sbezverk | sean-k-mooney: not all | 23:52 |
| sean-k-mooney | sbezverk: what will need to be done by k8 is adding the phyical interfaces | 23:53 |
| sbezverk | it will not created initial br-ex | 23:53 |
| sbezverk | which in kolla is done during bootstrap | 23:53 |
| sean-k-mooney | sbezverk: it will if you put it in the OVS_BRIDGE_MAPPINGS | 23:53 |
| sean-k-mooney | sbezverk: at least i think it does | 23:54 |
| sbezverk | sean-k-mmoney: hm it is not what I observed. if I do not have br-ex added to ovs neutron-opnevswitch-agent crashes | 23:54 |
| sbezverk | or exists with error code | 23:55 |
| sean-k-mooney | i will have to check but i know it will create the br-int if it does not exist | 23:55 |
| aNupoisc | while i was trying to install docker using above playbook I am hitting this error http://pastebin.com/m2Buvzw2 | 23:55 |
| sean-k-mooney | sbezverk: i taught it had the same logic to create br-ex | 23:55 |
| sbezverk | sean-k-mooney: yep br-int for sure | 23:55 |
| aNupoisc | it is unable to authenticate docker-engine ;) | 23:56 |
| sbezverk | unfortunately no, I hope it is a bug becuase if fixed it will simplify my life | 23:56 |
| jrich523 | ls | 23:57 |
| jrich523 | opps | 23:57 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
