| *** salv-orlando has joined #openstack-kolla | 00:02 | |
| SiRiuS | sdake_, | 00:07 |
|---|---|---|
| SiRiuS | +----+---------------------+-------+---------+ | 00:07 |
| SiRiuS | | ID | Hypervisor hostname | State | Status | | 00:07 |
| SiRiuS | +----+---------------------+-------+---------+ | 00:07 |
| SiRiuS | | 1 | controller01.local | up | enabled | | 00:07 |
| SiRiuS | +----+---------------------+-------+---------+ | 00:07 |
| sdake_ | looks good to me | 00:07 |
| SiRiuS | sdake_, http://paste.openstack.org/show/492849/ | 00:08 |
| SiRiuS | nova-api | 00:08 |
| SiRiuS | there is something about "Policy check for os_compute_api:os-hide-server-addresses failed..." | 00:09 |
| sbezverk | SiRiuS usually filtering removes all the hosts when you request some resources which openstack is not capable to provide | 00:09 |
| SiRiuS | sbezverk, any way to find out which resource it could not provide ? | 00:10 |
| *** achanda has quit IRC | 00:18 | |
| sbezverk | SiRiuS I usually do docker exec heka grep -r {failed instance id} /var/log/kolla | 00:20 |
| sbezverk | for sure some useful info to start troubleshooting.. | 00:20 |
| SiRiuS | sbezverk, I'm deploying with kolla-mesos :) | 00:21 |
| sbezverk | SiRiuS Sorry I did not know that, I have never played with mesos, so ignore what I said.. | 00:23 |
| SiRiuS | sbezverk np, thanks anyway | 00:23 |
| *** salv-orlando has quit IRC | 00:28 | |
| *** sdake has joined #openstack-kolla | 00:31 | |
| *** sdake_ has quit IRC | 00:34 | |
| *** vhosakot has joined #openstack-kolla | 00:39 | |
| *** achanda has joined #openstack-kolla | 00:57 | |
| *** achanda has quit IRC | 01:07 | |
| *** vhosakot has quit IRC | 01:11 | |
| *** sdake_ has joined #openstack-kolla | 01:17 | |
| *** sdake has quit IRC | 01:18 | |
| *** salv-orlando has joined #openstack-kolla | 01:27 | |
| sdake_ | manjeets around? | 01:29 |
| openstackgerrit | Steven Dake proposed openstack/kolla: Increase max scheduling attempts to 50 https://review.openstack.org/300914 | 01:30 |
| sdake_ | manjeets give that a go ^^ | 01:31 |
| *** SiRiuS has quit IRC | 01:33 | |
| *** salv-orlando has quit IRC | 01:44 | |
| *** iceyao has joined #openstack-kolla | 01:49 | |
| *** vhosakot has joined #openstack-kolla | 01:52 | |
| *** sdake has joined #openstack-kolla | 01:53 | |
| *** sdake_ has quit IRC | 01:54 | |
| *** vhosakot has quit IRC | 02:01 | |
| *** sdake has quit IRC | 02:03 | |
| *** sdake has joined #openstack-kolla | 02:06 | |
| *** achanda has joined #openstack-kolla | 02:07 | |
| *** vhosakot has joined #openstack-kolla | 02:10 | |
| *** achanda has quit IRC | 02:13 | |
| *** unicell has joined #openstack-kolla | 02:40 | |
| *** klint has joined #openstack-kolla | 02:44 | |
| *** unicell1 has quit IRC | 02:44 | |
| *** tfukushima has joined #openstack-kolla | 02:46 | |
| *** coolsvap has joined #openstack-kolla | 02:52 | |
| *** mdnadeem has joined #openstack-kolla | 03:02 | |
| *** salv-orlando has joined #openstack-kolla | 03:03 | |
| *** irfat has joined #openstack-kolla | 03:04 | |
| sdake | asalkeld ping | 03:06 |
| sdake | coolsvap ping | 03:06 |
| coolsvap | sdake: pong | 03:06 |
| sdake | coolsvap are you planning to finish the job on swift drop root | 03:06 |
| sdake | its been sitting in the review queue for months | 03:07 |
| sdake | if not uassign yourself | 03:07 |
| sdake | there is no shame in stepping back from work you can't do in the next week ;) | 03:07 |
| coolsvap | sdake: yes i was not working on it actively before rc2, but since then i am facing the issue with mysql-haproxy user, i am not able to test it | 03:07 |
| sdake | which issue is that | 03:08 |
| coolsvap | maybe i will unassign myself | 03:08 |
| sdake | is there a bug ? | 03:08 |
| coolsvap | yes there is | 03:08 |
| sdake | link? | 03:08 |
| coolsvap | there are workaround posted by vhosakot but its not working for me | 03:08 |
| coolsvap | just a sec | 03:08 |
| vhosakot | coolsvap: I think I recall.... link please | 03:09 |
| coolsvap | just a min | 03:09 |
| vhosakot | cool | 03:09 |
| sdake | lets either get you into a state where you can develop or get the work on someone elsees plate | 03:09 |
| sdake | i'd prefer option 1 ;) | 03:09 |
| *** achanda has joined #openstack-kolla | 03:09 | |
| sdake | sbezverk will do the drop root work if you are blocked on environmental issues | 03:10 |
| sdake | (I think ;) | 03:10 |
| sdake | he has a good knowledge of swift | 03:10 |
| coolsvap | i think i will unassign myslelf from the swift bug | 03:11 |
| vhosakot | coolsvap: https://bugs.launchpad.net/kolla/+bug/1491782 ? | 03:11 |
| openstack | Launchpad bug 1491782 in kolla "Fail to create haproxy mysql user" [Medium,Fix released] - Assigned to Sidharth Surana (ssurana) | 03:11 |
| sdake | we still need to solve your ability to develop | 03:12 |
| sdake | unless your locked up for the next week | 03:12 |
| sdake | because we hve a mountain of bugs | 03:12 |
| coolsvap | vhosakot: yes | 03:12 |
| vhosakot | coolsvap: the work around I mentioned in comment # 18 worked for me https://bugs.launchpad.net/kolla/+bug/1491782/comments/18 | 03:12 |
| vhosakot | coolsvap: did you try that /> | 03:12 |
| coolsvap | vhosakot: yes i did, but it did not work for me | 03:13 |
| vhosakot | ah ok | 03:13 |
| coolsvap | sdake: yes let me go through the bugs | 03:13 |
| *** salv-orlando has quit IRC | 03:13 | |
| coolsvap | for 2.0.0 milestone | 03:13 |
| sdake | comment #13 is the workaround | 03:14 |
| sdake | not workaround | 03:14 |
| sdake | the wayt o get it workign | 03:14 |
| sdake | tht needs to be documented | 03:14 |
| vhosakot | yes, comment #13 is the workwound... I just put all the commands and output in commeent #18 | 03:15 |
| coolsvap | sdake: yes its not working for me | 03:15 |
| sdake | comment #18 is disturbing | 03:16 |
| sdake | happening on 3/2 | 03:16 |
| *** achanda has quit IRC | 03:16 | |
| vhosakot | yes I did see the issue on 3/2.. we need to track this corner-case issue | 03:17 |
| sdake | i have opened this issue back up | 03:18 |
| sdake | and brought it back into the 2.0.0 release | 03:18 |
| sdake | mandre pingola | 03:22 |
| *** salv-orlando has joined #openstack-kolla | 03:22 | |
| sdake | coolsvap how mnay nodes do you have | 03:22 |
| sdake | just an aio? | 03:23 |
| coolsvap | sdake: aio | 03:23 |
| sdake | show me your /etc/hosts file | 03:23 |
| sdake | are you using centos binary or centos source? | 03:24 |
| coolsvap | centos binary | 03:25 |
| *** salv-orlando has quit IRC | 03:27 | |
| vhosakot | coolsvap: do you see the same error ? "Host is not allowed to connect to this MariaDB server" ? | 03:27 |
| coolsvap | vhosakot: yes | 03:27 |
| *** tfukushima has quit IRC | 03:28 | |
| *** sdake_ has joined #openstack-kolla | 03:29 | |
| sdake_ | coolsvap back | 03:31 |
| sdake_ | coolsnap show me docker images | fpaste | 03:31 |
| sdake_ | coolsvap that is ;) | 03:31 |
| *** sdake has quit IRC | 03:32 | |
| coolsvap | http://paste.fedoraproject.org/349189/74076214/ | 03:33 |
| sdake_ | your docker binary images are 3 days old | 03:33 |
| sdake_ | thats good - relativey recent images | 03:33 |
| sdake_ | trunk.rdoproject.org is down atm | 03:33 |
| coolsvap | yes i think i build them on friday | 03:34 |
| sdake_ | show me your globals.yaml file | 03:34 |
| sdake_ | which branch are you using, msater or mitaka? | 03:34 |
| coolsvap | master | 03:35 |
| sdake_ | run pip show kolla | fpaste | 03:35 |
| coolsvap | sdake_: pip show kolla http://paste.fedoraproject.org/349191/97409241 | 03:35 |
| sdake_ | ok problem #1 | 03:36 |
| sdake_ | you have kolla pip installed | 03:36 |
| sdake_ | sudo pip remove kolla | 03:36 |
| sdake_ | you are buliding old containers and using new anible | 03:36 |
| sdake_ | the build dsystem builds from /usr//include/kolla/docker first then the cwd next | 03:37 |
| sdake_ | it should be reverse of thta but is not | 03:37 |
| *** tfukushima has joined #openstack-kolla | 03:37 | |
| sdake_ | vhosakot can you change the docs to simply say pip install -r kolla/requirements.txt | 03:37 |
| coolsvap | sdake_: i pip install kolla everytime i update the repo on test stystem | 03:37 |
| sdake_ | oh ok well that should be fine then | 03:38 |
| sdake_ | maybe that shoudl e noted in the docs then | 03:38 |
| sdake_ | show me globals.yaml | 03:38 |
| sdake_ | getenforce | fpaste | 03:38 |
| coolsvap | globals -> http://paste.fedoraproject.org/349192/97409891/ | 03:38 |
| sdake_ | isn't .26 your dev box? | 03:39 |
| coolsvap | i think i got it | 03:39 |
| sdake_ | use haproxy | 03:39 |
| coolsvap | getenforce -> Enforcing | 03:39 |
| sdake_ | ya thats a problem | 03:39 |
| coolsvap | should it be permissive | 03:39 |
| sdake_ | you will have to redeploy | 03:39 |
| sdake_ | but isn't .26 your dev box? | 03:40 |
| sdake_ | internal_vip_address should be an unused ip address in your network | 03:40 |
| vhosakot | sdake_: is it a new step ? "pip install -r kolla/requirements.txt".... I always do "pip install kolla" after pulling master and it works fine ? "pip install kolla" wont work anymore and we need to do "pip install -r kolla/requirements.txt" ? | 03:40 |
| coolsvap | yes i might have done something here and there whilte trying | 03:40 |
| sdake_ | i dont hve the log of your hosts file, but iirc .26 was your host too | 03:40 |
| coolsvap | i will change it in globals and give it a try now | 03:40 |
| sdake_ | vhosakot "Note: If updating the repository via a git pull during development, make certain to run pip install again. Kolla builds containers from /usr/share/kolla first and the git working directory second. If you don't follow this step you will use old containers with new ansible resulting in undefined functionality" | 03:41 |
| sdake_ | or something simlar :) | 03:41 |
| coolsvap | yes | 03:41 |
| coolsvap | sdake_: yes | 03:42 |
| sdake_ | vip should not be your host ip | 03:42 |
| sdake_ | it has to be an unused ip address | 03:42 |
| sdake_ | that cupled with selinux | 03:42 |
| sdake_ | = bad day | 03:42 |
| sdake_ | if its in yur own house or something pick an unuse dip | 03:42 |
| sdake_ | i use 148 and 149 | 03:43 |
| sdake_ | for internal and external respectively | 03:43 |
| sdake_ | and centos binary no bueno | 03:43 |
| vhosakot | sdake_: here is what I do... "pip uninstall kolla" (this will delete everything /usr/share/kolla/), git pull origin master, pip install kolla (this will install new code in /usr/share/kolla/), build and deploy... | 03:43 |
| sdake_ | vhosakot sounds good - lets get it documented but explain WHY | 03:43 |
| coolsvap | sdake_: yes | 03:43 |
| sdake_ | this trips up everyone | 03:44 |
| sdake_ | becaue the devs susually dont pip insall ;) | 03:44 |
| sdake_ | I dont for instance | 03:44 |
| vhosakot | sdake_: I _always_ check the uptime of my images after building and befire deploying to make sure I deploy the latest container must master | 03:44 |
| sdake_ | vhosakot the age of your images only tells yu when they wer ebuilt | 03:44 |
| sdake_ | not if they wer ebuilt from the correct docker files | 03:44 |
| vhosakot | ah.. agreed | 03:44 |
| vhosakot | actually, I started doing pip install recently.. I was using code in repo to build and deploy.. will stick back to the old method... :) but, yeah, I make sure /usr/share/kolla/ is up-to-date.. | 03:46 |
| sdake_ | yup i liek tht model better | 03:47 |
| sdake_ | but lets get it documented ;) | 03:47 |
| sdake_ | it seriously trips up 90% of new users | 03:47 |
| sdake_ | all the osic intel folks hd this problem | 03:47 |
| sdake_ | so vikram | 03:48 |
| sdake_ | relating to your working around tha issue above | 03:48 |
| coolsvap | done its started :) | 03:48 |
| *** dave-mcc_ has quit IRC | 03:49 | |
| coolsvap | i mean passed the haproxy user step | 03:49 |
| coolsvap | brb | 03:49 |
| vhosakot | sdake_: yep.. | 03:49 |
| vhosakot | sdake_: did that workaround work ? | 03:49 |
| coolsvap | vhosakot: yes i think it worked + selinux = permissive | 03:49 |
| coolsvap | it was enforcing in my environment | 03:50 |
| vhosakot | coolsvap: so, you disabled selinux ? | 03:50 |
| coolsvap | no | 03:50 |
| coolsvap | made it permissive | 03:50 |
| vhosakot | ah | 03:50 |
| coolsvap | it was enforcing | 03:50 |
| sdake_ | regarindg comment #14in https://bugs.launchpad.net/kolla/+bug/1491782 | 03:50 |
| openstack | Launchpad bug 1491782 in kolla mitaka "Fail to create haproxy mysql user" [Critical,Triaged] | 03:50 |
| sdake_ | what was your host file? | 03:51 |
| sdake_ | was it like 127.0.0.1 inventory_hostname_ip? | 03:51 |
| sdake_ | Now, replace "127.0.1.1 kolla-ubuntu" with "192.168.122.201 kolla-ubuntu" in /etc/hosts | 03:52 |
| sdake_ | this needs a doc fix | 03:54 |
| sdake_ | i don't understand why 127.0.1.1 gets set | 03:54 |
| sdake_ | the intel osic g uys had htis problem too | 03:54 |
| sdake_ | anything in the inventory file should never have a non valid ip address | 03:55 |
| openstackgerrit | Vikram Hosakote proposed openstack/kolla: Add general_log to mariadb container and make heka collect it https://review.openstack.org/299133 | 03:56 |
| sdake_ | what is general log | 03:56 |
| vhosakot | sdake_: ok, at the time the work-around fixed the issue, I changed "127.0.1.1 kolla-ubuntu" to "192.168.122.201 kolla-ubuntu".... 192.168.122.201 is not kolla_internal_vip_address and is the mgmt IP of my dev VM... | 04:00 |
| sdake_ | in the database or in the /etc/hsots file? | 04:00 |
| vhosakot | in /etc/hosts... it looked like this | 04:01 |
| vhosakot | 127.0.0.1localhost | 04:01 |
| vhosakot | 192.168.122.201 kolla-ubuntu | 04:01 |
| sdake_ | what is 201? | 04:01 |
| sdake_ | the ip of eth0? | 04:01 |
| vhosakot | yes.. ip of eth0 192.168.122.201 | 04:01 |
| sdake_ | where is 127.0.1.1 coming from? | 04:02 |
| vhosakot | the same IP i use to SSH into my kolla dev machine | 04:02 |
| vhosakot | don't Ubuntu put it for me that day when I saw the issue | 04:02 |
| vhosakot | coolsvap: can you psate your/etc/hosts here please | 04:02 |
| vhosakot | sdake_: general log mariadb log... mariadb logs are not collected at all, just mariadb's error logs are collected.... https://bugs.launchpad.net/kolla/+bug/1563668 | 04:03 |
| openstack | Launchpad bug 1563668 in kolla "Add general_log to mariadb container and make heka collect it" [High,In progress] - Assigned to Vikram Hosakote (vhosakot) | 04:03 |
| vhosakot | if coolsvap ack's it, it could be merged soon | 04:04 |
| vhosakot | test results are in http://paste.openstack.org/show/492852/ | 04:04 |
| sdake_ | It doesn't really mean anything in particular (it's just another IP in the 127.0.0.0/8 block). | 04:04 |
| sdake_ | The reason it's used as the IP for your hostname is explained in Section 10.4 of the Debian Reference Manual. | 04:04 |
| sdake_ | Quote: | 04:04 |
| sdake_ | Some software (e.g., GNOME) expects the system hostname to be resolvable to an IP address with a canonical fully qualified domain name. This is really improper because system hostnames and domain names are two very different things; but there you have it. In order to support that software, it is necessary to ensure that the system hostname can be resolved. Most often this is done by putting a line in /etc/hosts containing some IP | 04:04 |
| sdake_ | address and the system hostname. If your system has a permanent IP address then use that; otherwise use the address 127.0.1.1. | 04:04 |
| sdake_ | that explains why ubuntu foks hve this 127.0.1.1 problem | 04:05 |
| sdake_ | this doesn't happen on centos | 04:05 |
| sdake_ | what is happening is | 04:06 |
| vhosakot | ah | 04:06 |
| sdake_ | if the hostname is abc | 04:06 |
| vhosakot | yeah... | 04:06 |
| sdake_ | 127.0.1.1 abc abc.localnet is being set | 04:06 |
| sdake_ | then when facts are gathered by ansible, abc is 127.0.1.1 | 04:06 |
| sdake_ | which is clearly wrong | 04:06 |
| vhosakot | yess... | 04:06 |
| sdake_ | the problem with 127.0.1.1 is that it is global acros teh cluster since its in /etc/hsots | 04:07 |
| vhosakot | should it be "192.168.122.201 abc" correct then ? | 04:07 |
| sdake_ | yes it should | 04:07 |
| sdake_ | so this is a user error problem | 04:07 |
| sdake_ | which can be fixed by proper documentation | 04:07 |
| vhosakot | sdake_: we need to document it... | 04:07 |
| vhosakot | agreed user / operator error | 04:07 |
| vhosakot | sdake_: we could document so user knows | 04:07 |
| sdake_ | agree | 04:08 |
| vhosakot | qsg ? | 04:08 |
| sdake_ | yup | 04:08 |
| sdake_ | every host in the cluster should be resolvable by dns or the /etc/host file | 04:08 |
| sdake_ | and they should all be unique, meaning 127.0.1.1 is not one of them | 04:08 |
| sdake_ | i'd give a link to the ubuntu manual where ubuntu breaks things by doing what it is doing | 04:08 |
| sdake_ | if tis installer was more intellgient it would put the actual ip address in etchosts | 04:09 |
| vhosakot | what should I write ? "If you are using Ubuntu 14.04 and you see "127.0.1.1 <hostname>", replace it with "<IP address of eth0> <hostname>""... is this correct ? | 04:09 |
| sdake_ | yup | 04:09 |
| sdake_ | also you need to explain the use dns or hosts file to put all nodes in the inventory into the hosts file or be resolvable via dns | 04:10 |
| vhosakot | https://www.debian.org/doc/manuals/debian-reference/ch05.en.html#_the_hostname_resolution ? | 04:11 |
| *** sdake_ is now known as sdake | 04:12 | |
| vhosakot | sdake_: meaning all the hosts in the kolla inventory file must be put in /etc/hosts on the deploy node with their IP addresses ? | 04:12 |
| sdake | on all nodes including the deploy node | 04:12 |
| *** achanda has joined #openstack-kolla | 04:13 | |
| vhosakot | ok, put the hostnames and IP addresses of all the hosts in the inventory file in/etc/hosts on all the hosts including the deploy node.. correct ? | 04:14 |
| *** achanda has quit IRC | 04:18 | |
| vhosakot | coolsvap: did you see the issue on Ubuntu ? can you paste your /etc/hosts | 04:19 |
| vhosakot | is http://trunk.rdoproject.org down today ? | 04:20 |
| vhosakot | coolsvap: ping | 04:22 |
| coolsvap | vhosakot: sorry was away for a bit | 04:25 |
| coolsvap | no i didnt see it on ubuntu | 04:25 |
| vhosakot | cool, np | 04:25 |
| coolsvap | i have only centos env running now | 04:25 |
| vhosakot | cool | 04:25 |
| coolsvap | i will check and let you know | 04:25 |
| vhosakot | can you paste your /etc/hosts | 04:25 |
| *** vhosakot has quit IRC | 04:31 | |
| *** jasonsb has joined #openstack-kolla | 04:36 | |
| *** salv-orlando has joined #openstack-kolla | 04:42 | |
| *** salv-orlando has quit IRC | 04:45 | |
| *** achanda has joined #openstack-kolla | 04:46 | |
| *** Marga_ has joined #openstack-kolla | 04:55 | |
| *** jasonsb has quit IRC | 05:09 | |
| *** jasonsb has joined #openstack-kolla | 05:13 | |
| *** vhosakot has joined #openstack-kolla | 05:15 | |
| sdake | coolsvap note rdoproject is dead | 05:19 |
| coolsvap | sdake: yes i read the outage notification just now | 05:20 |
| coolsvap | i am not building anything now | 05:20 |
| coolsvap | i am looking at adding the SELINUX and /etc/hosts checks to prechecks | 05:20 |
| vhosakot | yep, trunk rdo is down... the gate in my PS failed | 05:20 |
| *** jasonsb has quit IRC | 05:24 | |
| sdake | coolsvap did you get your environment operational again | 05:25 |
| sdake | vhosakot we will have to ignore the centos gates or the moment | 05:25 |
| vhosakot | ok cool | 05:25 |
| coolsvap | sdake: yes almost | 05:25 |
| vhosakot | sdake: I'm almost done with adding precehck to check empty passwrods in /etc/kolla/passwords... will push PS soon | 05:25 |
| sdake | coolsvap don't forget generate_passwords.yml ;-) | 05:26 |
| coolsvap | yes | 05:26 |
| *** jasonsb has joined #openstack-kolla | 05:27 | |
| coolsvap | its not generating passwords | 05:29 |
| coolsvap | :-/ | 05:29 |
| sdake | the password file has to b eempty | 05:29 |
| sdake | or have unset psswordsi mean | 05:29 |
| *** reddy has joined #openstack-kolla | 05:34 | |
| sdake | vhosakot hae you ever run with enable_tls? | 05:46 |
| vhosakot | no... why ? | 05:46 |
| sdake | ca nyou try it | 05:46 |
| sdake | and see if nova boot works for you | 05:46 |
| sdake | i wnt to see if its my environment only | 05:46 |
| sdake | kolla-ansbile certificates | 05:47 |
| sdake | this creates a haproxy-ca.crt file | 05:47 |
| sdake | add this to your openrc: | 05:47 |
| sdake | export OS_CACERT=/Users/sdake/demo/haproxy-ca.crt | 05:47 |
| vhosakot | pull latest master, build images, kolla-ansbile certificates, deploy, export OS_CACERT=/Users/sdake/demo/haproxy-ca.crt, boot nova VM.. are these steps right ? | 05:48 |
| sdake | you need to copy os_cacrt from your deploy target to your client node | 05:48 |
| sdake | and set the path correctly | 05:48 |
| sdake | you can skip the image buliding | 05:49 |
| vhosakot | ah ok... | 05:49 |
| sdake | and ksip pull latet master | 05:49 |
| sdake | nothing hs gone in since we tagged | 05:49 |
| vhosakot | cool, let me try it | 05:49 |
| sdake | dont forget enable tls in config file too ;) | 05:50 |
| vhosakot | yep, will have kolla_enable_tls_external: "yes" in globals.yml :) | 05:53 |
| sdake | disabling tls lets my vms go to active state | 05:53 |
| vhosakot | do I just re-deploy or, clean all my containers and deploy ? | 05:54 |
| sdake | good news is nova works :) | 05:54 |
| sdake | clean all containers | 05:55 |
| sdake | clean all images | 05:55 |
| sdake | redeploy from reiistry | 05:55 |
| sdake | is what i do | 05:55 |
| vhosakot | cool... will re-deploy | 05:55 |
| *** iceyao_ has joined #openstack-kolla | 05:57 | |
| *** iceyao has quit IRC | 05:57 | |
| sdake | https://bugs.launchpad.net/kolla/+bug/1565624 | 05:58 |
| openstack | Launchpad bug 1565624 in kolla mitaka "with tls enabled, vms never enter started state" [Critical,Triaged] | 05:58 |
| sdake | I am pretty sure i demoed tls and it worked for nova boot a few weeks ago | 05:58 |
| openstackgerrit | Vikram Hosakote proposed openstack/kolla: Check empty passwords in /etc/kolla/passwords.yml in prechecks https://review.openstack.org/300945 | 05:59 |
| vhosakot | sdake: ^^^ I've tested it | 06:00 |
| sdake | the docker registry password shoudl be empty | 06:01 |
| sdake | does your patch deal with that | 06:02 |
| sdake | should/could | 06:02 |
| vhosakot | yep... it is not empty.. it is null | 06:02 |
| vhosakot | after running kolla-genpwd, I see docker_registry_password: null in passwords.yml.... is this right ? | 06:03 |
| sdake | i don't know | 06:04 |
| sdake | must be ;) | 06:04 |
| *** SiRiuS has joined #openstack-kolla | 06:05 | |
| sdake | if it works | 06:05 |
| vhosakot | I and ccesario both see null for docker_registry_password after running kolla-genpwd :) | 06:05 |
| vhosakot | thanks for acking :) | 06:05 |
| *** vhosakot has quit IRC | 06:07 | |
| sdake | so ssl | 06:10 |
| sdake | where ya at ;) | 06:10 |
| sdake | isnt an admin port public? | 06:13 |
| *** salv-orlando has joined #openstack-kolla | 06:13 | |
| *** salv-orlando has quit IRC | 06:16 | |
| *** SiRiuS has quit IRC | 06:22 | |
| sdake | appears not | 06:22 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Add check for SELINUX in prechecks https://review.openstack.org/300955 | 06:25 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Add check for SELINUX in prechecks https://review.openstack.org/300955 | 06:30 |
| *** mbound has joined #openstack-kolla | 06:32 | |
| *** SiRiuS has joined #openstack-kolla | 06:32 | |
| *** mbound has quit IRC | 06:32 | |
| *** Marga_ has quit IRC | 06:33 | |
| *** Marga_ has joined #openstack-kolla | 06:33 | |
| sdake | i just had anothe inspection of dave's ssl patch and i see no defects | 06:35 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Add check for SELINUX in prechecks https://review.openstack.org/300955 | 06:47 |
| *** achanda has quit IRC | 06:48 | |
| *** sdake has quit IRC | 06:48 | |
| *** sdake has joined #openstack-kolla | 06:50 | |
| *** Serlex has joined #openstack-kolla | 06:51 | |
| *** sdake_ has joined #openstack-kolla | 06:54 | |
| *** sdake has quit IRC | 06:56 | |
| *** Serlex has quit IRC | 06:56 | |
| *** Serlex has joined #openstack-kolla | 06:57 | |
| *** sdake has joined #openstack-kolla | 06:59 | |
| *** sdake_ has quit IRC | 07:00 | |
| *** achanda has joined #openstack-kolla | 07:03 | |
| *** mikelk has joined #openstack-kolla | 07:04 | |
| *** cristicalin has joined #openstack-kolla | 07:05 | |
| *** salv-orlando has joined #openstack-kolla | 07:17 | |
| *** achanda has quit IRC | 07:24 | |
| *** achanda has joined #openstack-kolla | 07:24 | |
| *** jmccarthy has joined #openstack-kolla | 07:26 | |
| *** jmccarthy has left #openstack-kolla | 07:27 | |
| *** cristicalin has quit IRC | 07:29 | |
| *** cristicalin has joined #openstack-kolla | 07:30 | |
| *** achanda has quit IRC | 07:32 | |
| *** mdnadeem has quit IRC | 07:37 | |
| *** Jeffrey4l__ has quit IRC | 07:40 | |
| *** cristicalin has quit IRC | 07:40 | |
| *** jmccarthy has joined #openstack-kolla | 07:40 | |
| *** Serlex has quit IRC | 07:40 | |
| *** Jeffrey4l__ has joined #openstack-kolla | 07:43 | |
| *** Serlex has joined #openstack-kolla | 07:44 | |
| *** salv-orlando has quit IRC | 07:53 | |
| *** sdake has quit IRC | 07:53 | |
| *** mdnadeem has joined #openstack-kolla | 07:54 | |
| *** salv-orlando has joined #openstack-kolla | 07:56 | |
| *** mgoddard has joined #openstack-kolla | 07:59 | |
| *** athomas has joined #openstack-kolla | 08:12 | |
| *** gfidente has joined #openstack-kolla | 08:16 | |
| *** iceyao_ has quit IRC | 08:32 | |
| *** achanda has joined #openstack-kolla | 08:32 | |
| *** ccesario_ has joined #openstack-kolla | 08:33 | |
| *** achanda has quit IRC | 08:36 | |
| *** mbound has joined #openstack-kolla | 08:42 | |
| *** athomas has quit IRC | 08:45 | |
| *** pbourke has joined #openstack-kolla | 08:51 | |
| *** mbound has quit IRC | 08:55 | |
| *** ccesario has quit IRC | 08:57 | |
| *** ccesario has joined #openstack-kolla | 08:57 | |
| *** wznoinsk_ has joined #openstack-kolla | 09:03 | |
| *** mbound has joined #openstack-kolla | 09:04 | |
| *** athomas has joined #openstack-kolla | 09:04 | |
| *** wznoinsk_ is now known as wznoinsk | 09:04 | |
| openstackgerrit | Merged openstack/kolla: Check empty passwords in /etc/kolla/passwords.yml in prechecks https://review.openstack.org/300945 | 09:26 |
| openstackgerrit | Paul Bourke proposed openstack/kolla: Debug oraclelinux deploy gates https://review.openstack.org/300988 | 09:26 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Check empty passwords in /etc/kolla/passwords.yml in prechecks https://review.openstack.org/300991 | 09:38 |
| *** SiRiuS has quit IRC | 09:40 | |
| openstackgerrit | Merged openstack/kolla: Implement nova-ssh container https://review.openstack.org/297963 | 09:50 |
| *** reddy has quit IRC | 09:58 | |
| *** achanda has joined #openstack-kolla | 10:05 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Correct typo https://review.openstack.org/301009 | 10:09 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Implement nova-ssh container https://review.openstack.org/301010 | 10:10 |
| *** achanda has quit IRC | 10:11 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Implement nova-ssh container https://review.openstack.org/301010 | 10:12 |
| *** reddy has joined #openstack-kolla | 10:14 | |
| *** ccesario_ has quit IRC | 10:19 | |
| *** rhallisey has joined #openstack-kolla | 10:19 | |
| *** salv-orl_ has joined #openstack-kolla | 10:30 | |
| *** salv-orlando has quit IRC | 10:32 | |
| *** mikelk has quit IRC | 10:38 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/kolla: Fix /etc/hosts in gate https://review.openstack.org/297247 | 10:41 |
| *** openstackgerrit has quit IRC | 10:48 | |
| *** tfukushima has quit IRC | 10:48 | |
| *** rhallisey has quit IRC | 10:48 | |
| *** openstackgerrit has joined #openstack-kolla | 10:48 | |
| openstackgerrit | Merged openstack/kolla: Add general_log to mariadb container and make heka collect it https://review.openstack.org/299133 | 10:51 |
| *** ccesario_ has joined #openstack-kolla | 10:56 | |
| *** rhallisey has joined #openstack-kolla | 11:06 | |
| *** rhallisey has quit IRC | 11:12 | |
| *** ccesario_ has quit IRC | 11:17 | |
| *** ccesario_ has joined #openstack-kolla | 11:23 | |
| *** rhallisey has joined #openstack-kolla | 11:31 | |
| *** sdake has joined #openstack-kolla | 11:34 | |
| *** rhallisey has quit IRC | 11:35 | |
| *** sdake_ has joined #openstack-kolla | 11:36 | |
| sdake_ | morning | 11:37 |
| *** sdake has quit IRC | 11:38 | |
| *** iceyao has joined #openstack-kolla | 11:43 | |
| *** rhallisey has joined #openstack-kolla | 11:46 | |
| *** ccesario_ has quit IRC | 11:54 | |
| *** rhallisey has quit IRC | 11:55 | |
| *** mlima has joined #openstack-kolla | 11:58 | |
| sbezverk | good morning | 11:58 |
| ccesario | morning guys! | 11:58 |
| ccesario | sdake_, fixed :P https://review.openstack.org/#/c/300896/ | 12:05 |
| patchbot | ccesario: patch 300896 - kolla - Make Heka write Keepalived logs to dedicated direc... | 12:05 |
| *** reddy has quit IRC | 12:06 | |
| sdake_ | ccesario nice | 12:06 |
| sdake_ | sbezverk can you take over drop root on swift plz | 12:06 |
| sdake_ | its critical | 12:06 |
| sdake_ | i spke with coolsvap 3-4 horus ago when i was asleep and he said he would unassig himself | 12:06 |
| coolsvap | sdake_: i have unassigned myself | 12:07 |
| sbezverk | sdake_ sure, just let me know bug id | 12:07 |
| coolsvap | sbezverk just a min | 12:08 |
| coolsvap | sbezverk: https://bugs.launchpad.net/kolla/+bug/1553895 | 12:08 |
| openstack | Launchpad bug 1553895 in kolla "Drop root for swift" [Critical,In progress] | 12:08 |
| *** achanda has joined #openstack-kolla | 12:09 | |
| coolsvap | i have a patch https://review.openstack.org/259374 | 12:09 |
| coolsvap | but i was not able to test it | 12:09 |
| sdake_ | coolsvap ya start from coolsvap's existing work | 12:09 |
| sdake_ | rathe sbezverk ya start rom coolsvap's existing work | 12:09 |
| * coolsvap will be away for sometime | 12:11 | |
| *** rhallisey has joined #openstack-kolla | 12:13 | |
| *** achanda has quit IRC | 12:14 | |
| sbezverk | sdake the primary target for this patch stable/mitaka or newton? | 12:16 |
| *** rhallisey has quit IRC | 12:17 | |
| *** rhallisey has joined #openstack-kolla | 12:22 | |
| *** reddy has joined #openstack-kolla | 12:22 | |
| *** mdnadeem has quit IRC | 12:22 | |
| *** rhallisey has quit IRC | 12:26 | |
| *** sdake_ has quit IRC | 12:30 | |
| *** salv-orl_ has quit IRC | 12:33 | |
| *** sdake has joined #openstack-kolla | 12:34 | |
| *** salv-orlando has joined #openstack-kolla | 12:34 | |
| sdake | https://drive.google.com/file/d/0B8q6xDPETSkHcGNpdDVHbEpmcmc/view?usp=sharing | 12:36 |
| sdake | office remodel mission complete | 12:36 |
| sdake | sbezverk right, you have one week to gt it done ;) | 12:37 |
| *** salv-orlando has quit IRC | 12:38 | |
| sbezverk | sdake what a weird amplifier you have, is it like from 1960 era ;-) ?? | 12:38 |
| sdake | no its modern | 12:38 |
| sbezverk | sdake it was attempt to joke, I know | 12:39 |
| sdake | http://www.stevehuffphoto.com/line-magnetic-219ia-integrated-tube-amp-review-300b-and-845-tube-magic/ | 12:39 |
| sdake | ts a 300b set tube amp so the design is from the 30s | 12:39 |
| sdake | but its been obviosuy imrpvoed over the last 100 yeares | 12:39 |
| sdake | its a dual mono amp which is winning :) | 12:39 |
| sdake | my dac (thing on the left) is line magnetic as ewell | 12:40 |
| sdake | turntable is dr. feickert | 12:40 |
| *** mbound has quit IRC | 12:40 | |
| sdake | the other thign on the left up top is my phonoe preamp | 12:40 |
| *** stvnoyes has quit IRC | 12:41 | |
| sdake | i just put it togtether now but was not energeized enough at 6am to plug the wires in to see if it all sitll works ;) | 12:41 |
| sbezverk | sdake you must be a real music fanatic ;-) | 12:45 |
| *** dwalsh has joined #openstack-kolla | 12:47 | |
| *** dwalsh has quit IRC | 12:48 | |
| *** dwalsh has joined #openstack-kolla | 12:49 | |
| *** prithiv has joined #openstack-kolla | 12:49 | |
| *** mlima has quit IRC | 12:49 | |
| *** Marga_ has quit IRC | 12:51 | |
| sdake | sbezverk money to burn - women or drugs | 12:51 |
| sdake | or stereo equipment | 12:51 |
| sdake | thats a line rom jay leno btw, I didn't originate it :) | 12:51 |
| sbezverk | sdake have you had enough coffee to chat about qemu versions conflict? | 12:53 |
| elmiko | sdake: awesome stereo setup +1 | 12:53 |
| sdake | sbezverk no i am writing user stories | 12:55 |
| sdake | until 11am | 12:55 |
| sdake | then have a cutomer meeting | 12:55 |
| sdake | then have a user story ureview | 12:55 |
| sdake | then i'm all yours for about an 1-2 | 12:55 |
| sdake | sounds goo? | 12:55 |
| sdake | elmiko thanks :) | 12:55 |
| sdake | sbezverk also i am on a caffeine free diet for the most part | 12:56 |
| sdake | the most part being when i cheat and drink a coke | 12:56 |
| *** klint has quit IRC | 12:58 | |
| sbezverk | sdake got it | 12:58 |
| *** SiRiuS has joined #openstack-kolla | 12:58 | |
| sbezverk | sdake recent studies claim that 2-3 cups of coffee a day protect from alzheimer | 13:00 |
| *** rhallisey has joined #openstack-kolla | 13:02 | |
| *** rhallisey has quit IRC | 13:07 | |
| *** rhallisey has joined #openstack-kolla | 13:08 | |
| *** dougs has joined #openstack-kolla | 13:10 | |
| *** salv-orlando has joined #openstack-kolla | 13:11 | |
| *** rhallisey has quit IRC | 13:13 | |
| *** mbound has joined #openstack-kolla | 13:14 | |
| *** Marga_ has joined #openstack-kolla | 13:18 | |
| *** mbound_ has joined #openstack-kolla | 13:19 | |
| *** rhallisey_ has joined #openstack-kolla | 13:19 | |
| *** mbound has quit IRC | 13:22 | |
| *** prithiv has quit IRC | 13:23 | |
| *** prithiv has joined #openstack-kolla | 13:23 | |
| *** kproskurin has joined #openstack-kolla | 13:24 | |
| *** vhosakot has joined #openstack-kolla | 13:27 | |
| *** reddy has quit IRC | 13:28 | |
| *** mlima has joined #openstack-kolla | 13:29 | |
| *** kproskurin has quit IRC | 13:30 | |
| *** mikelk has joined #openstack-kolla | 13:38 | |
| *** mgoddard_ has joined #openstack-kolla | 13:45 | |
| *** mgoddard has quit IRC | 13:48 | |
| Serlex | what happened to #kolla | 13:51 |
| *** mgoddard_ has quit IRC | 13:55 | |
| *** mgoddard has joined #openstack-kolla | 13:55 | |
| vhosakot | Serlex: renamed / moved to #openstack-kolla | 13:56 |
| Serlex | lol which is why i'm here i guess | 13:57 |
| *** reddy has joined #openstack-kolla | 13:58 | |
| *** andrewvwebber has joined #openstack-kolla | 13:58 | |
| *** mbound_ has quit IRC | 13:59 | |
| vhosakot | yep :) /join #kolla redirects here | 14:00 |
| *** Marga_ has quit IRC | 14:02 | |
| ccesario | vhosakot, ping ? | 14:05 |
| vhosakot | ccesario: ping | 14:05 |
| ccesario | vhosakot, :) .... any tip about https://review.openstack.org/#/c/297984/ | 14:06 |
| patchbot | ccesario: patch 297984 - kolla - Added hostname param into kolla_docker | 14:06 |
| *** achanda has joined #openstack-kolla | 14:12 | |
| *** achanda has quit IRC | 14:17 | |
| *** mbound has joined #openstack-kolla | 14:23 | |
| *** jasonsb has quit IRC | 14:25 | |
| *** jasonsb has joined #openstack-kolla | 14:26 | |
| *** haplo37 has joined #openstack-kolla | 14:27 | |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Use debian repos for debian base docker image https://review.openstack.org/288936 | 14:30 |
| *** inc0 has joined #openstack-kolla | 14:31 | |
| *** achanda has joined #openstack-kolla | 14:31 | |
| inc0 | hey guys | 14:32 |
| inc0 | sbezverk, around? | 14:32 |
| *** banix has joined #openstack-kolla | 14:32 | |
| *** Jeffrey4l__ has quit IRC | 14:33 | |
| sbezverk | inc0 yep | 14:34 |
| inc0 | so issue with iscsi is mounting /sys/kernel/config in rw mode | 14:35 |
| sbezverk | inc0 unfortunately it is required as iscsd stores targets there | 14:35 |
| inc0 | after turning it to ro it will deploy iscsi incorrectly, but will not fail | 14:35 |
| inc0 | I would assume that's the hard part why we don't have iscsi just yet;) | 14:36 |
| sbezverk | inc0 it is working fine on centos | 14:37 |
| sbezverk | I guess there are some flavor specific particularities.. | 14:37 |
| inc0 | maybe | 14:38 |
| *** achanda has quit IRC | 14:38 | |
| sbezverk | inc0 I will try to find ubuntu server to test, unfortunately all I have so far are centos and red hat.. on both it works 100% | 14:40 |
| inc0 | I just deploy aio on vm on buntu 15.10 | 14:40 |
| inc0 | and that fails | 14:40 |
| sbezverk | inc0 could you share some logs, it will help to start thinking in the right direction. | 14:41 |
| *** rmart04 has joined #openstack-kolla | 14:42 | |
| inc0 | sbezverk, sure, let me up my log level and paste it somewhere | 14:47 |
| sbezverk | inc0 you have my email, right if you could send it there would be great | 14:48 |
| inc0 | I'll post it in review for reference for others | 14:49 |
| inc0 | maybe we can have bigger discussion about this stuff as it would be great to enable iscsi | 14:50 |
| sbezverk | inc0 absolutely I am tired of cherry picking iscsi patch to get my test bed going with lvm2/iscsi ;-) | 14:55 |
| *** iceyao has quit IRC | 14:56 | |
| inc0 | I'll do full rebuild so it will take some time | 14:56 |
| *** mlima has quit IRC | 15:00 | |
| *** daneyon has joined #openstack-kolla | 15:00 | |
| openstackgerrit | Merged openstack/kolla: Correct typo https://review.openstack.org/301009 | 15:03 |
| *** daneyon_ has joined #openstack-kolla | 15:04 | |
| *** Serlex has quit IRC | 15:04 | |
| *** daneyon has quit IRC | 15:07 | |
| openstackgerrit | Andrey Pavlov proposed openstack/kolla-mesos: Fix "kolla-mesos chronos show" https://review.openstack.org/301178 | 15:09 |
| *** mlima has joined #openstack-kolla | 15:15 | |
| *** prithiv has quit IRC | 15:18 | |
| *** prithiv has joined #openstack-kolla | 15:22 | |
| *** daneyon_ has quit IRC | 15:22 | |
| *** daneyon has joined #openstack-kolla | 15:23 | |
| *** mikelk has quit IRC | 15:25 | |
| *** stvnoyes has joined #openstack-kolla | 15:29 | |
| *** stvnoyes has left #openstack-kolla | 15:29 | |
| ccesario | vhosakot, regarding ps https://review.openstack.org/#/c/300945/ ... is it possible make it check ${CONFIG_DIR} (set in tools/kolla-ansible) instead fixed path /etc/kolla ..!? | 15:30 |
| patchbot | ccesario: patch 300945 - kolla - Check empty passwords in /etc/kolla/passwords.yml ... (MERGED) | 15:30 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Use debian repos for debian base docker image https://review.openstack.org/288936 | 15:34 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: use silent option for curl download in heka container https://review.openstack.org/297739 | 15:35 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make container dind build with debian https://review.openstack.org/297434 | 15:36 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make heka container build with debian https://review.openstack.org/297737 | 15:37 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make kibana container build with debian https://review.openstack.org/297206 | 15:37 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make marathon container build with debian https://review.openstack.org/297378 | 15:37 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make mesos-slave container build with debian https://review.openstack.org/297385 | 15:38 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make mesosphere-base build with debian https://review.openstack.org/297358 | 15:38 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make nova-libvirt container build with debian https://review.openstack.org/297169 | 15:39 |
| vhosakot | ccesario: yes, good point.. I will fix it to check the path in CONFIG_DIR.. will submit patch set soon | 15:39 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: make rabbitmq container build with debian https://review.openstack.org/297728 | 15:39 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make swift-base container build with debian https://review.openstack.org/297121 | 15:40 |
| openstackgerrit | Benedikt Trefzer proposed openstack/kolla: Make container kolla-toolbox build with debian https://review.openstack.org/297104 | 15:40 |
| *** salv-orlando has quit IRC | 15:42 | |
| *** reddy has quit IRC | 15:44 | |
| *** joliver has joined #openstack-kolla | 15:46 | |
| sdake | is mandre around | 15:48 |
| *** rhallisey_ has quit IRC | 15:54 | |
| gmmaha | inc0: good morning | 15:54 |
| inc0 | hey gmmaha what's up? | 15:55 |
| gmmaha | inc0: any example of the TrivialFix tag? | 15:55 |
| gmmaha | i am trying to look through gerrit to see if i can spot one, but thought i might as well ask | 15:55 |
| gmmaha | :) | 15:55 |
| inc0 | gmmaha, just add TrivialFix to commit message | 15:55 |
| gmmaha | inc0: aah | 15:55 |
| gmmaha | ok | 15:55 |
| gmmaha | thanks | 15:55 |
| inc0 | this is kolla specific I think | 15:56 |
| inc0 | so you won't need to make bug for a 2 lines of docs add | 15:56 |
| gmmaha | inc0: aaah gotcha. adding it now | 15:56 |
| sdake | gmmaha | 15:57 |
| sdake | can you pign mandre | 15:57 |
| sdake | manjeets | 15:57 |
| sdake | i need him to test a bug ffix | 15:57 |
| sdake | asap | 15:57 |
| gmmaha | sdake: for sure.. let me go and find him | 15:57 |
| gmmaha | sdake: he isnt around.. not able to find him online yet | 15:57 |
| gmmaha | he should be in anytime.. | 15:58 |
| gmmaha | is this something i can help you test? | 15:58 |
| sdake | it requries his gear | 15:58 |
| gmmaha | aaah | 15:58 |
| sdake | https://bugs.launchpad.net/kolla/+bug/1563664 | 15:58 |
| openstack | Launchpad bug 1563664 in kolla "creating 100 vms via horizon on 64 node cluster returns error "max scheduling attempts reached 3"" [Critical,In progress] - Assigned to Steven Dake (sdake) | 15:58 |
| gmmaha | will let him know you are looking for him | 15:58 |
| sdake | ask him to read thatbug log | 15:59 |
| sdake | it has specific instructions | 15:59 |
| sdake | i may be in meetings htis morning | 15:59 |
| gmmaha | sdake: will definitely do | 15:59 |
| *** banix has quit IRC | 15:59 | |
| sdake | thanks gmmaha :) | 15:59 |
| sdake | anything your blocked on atm? | 15:59 |
| sdake | sbezverk ok lets chat now | 15:59 |
| gmmaha | sdake: i am good for now.. just to figure outhow to get UEFI images to boot with openstack :) | 15:59 |
| sdake | that i can't help yo uwith | 16:00 |
| *** sean-k-mooney has quit IRC | 16:00 | |
| gmmaha | sdake: :) i wouldnt burden you with that either | 16:00 |
| gmmaha | its something i need to solve | 16:00 |
| openstackgerrit | Ganesh Maharaj Mahalingam proposed openstack/kolla: Add comments on site.yml neutron hosts reordering https://review.openstack.org/300651 | 16:01 |
| *** rhallisey has joined #openstack-kolla | 16:01 | |
| gmmaha | inc0: ^^ updated the patch.. Let me know if that looks fine | 16:01 |
| sbezverk | sdake the problem I see is ceph pushes some restrictions on version of qemu | 16:06 |
| inc0 | sbezverk, http://paste.openstack.org/show/492894/ | 16:06 |
| sbezverk | sdake I think it might create in future some problems because running things on qemu 1.5.3 cannot last long | 16:07 |
| sbezverk | inc0 I really doubt I have "mkdir /sys/kernel/config" anywhere | 16:08 |
| sbezverk | inc0 I am not sure why ubuntu wants to create this folder when it is already there | 16:09 |
| inc0 | yeah that the issue...not on ubuntu | 16:10 |
| inc0 | probably iscsi is keeping it's stuff someplace else | 16:10 |
| inc0 | on ubuntu | 16:10 |
| vhosakot | ccesario: ping | 16:10 |
| sbezverk | sdake it would be great to be able to turn off, some components which are not required for some people | 16:10 |
| inc0 | heh...conditional includes all the way:) | 16:10 |
| *** jmccarthy has left #openstack-kolla | 16:11 | |
| * manjeets is on his desk sdake | 16:11 | |
| vhosakot | ccesario: can you send me your value of $CONFIG_DIR in kolla-ansible script... I need it to test... | 16:11 |
| inc0 | sbezverk, it seems we need to figure out conditional mounts for this | 16:11 |
| gmmaha | manjeets: check https://bugs.launchpad.net/kolla/+bug/1563664 | 16:11 |
| openstack | Launchpad bug 1563664 in kolla "creating 100 vms via horizon on 64 node cluster returns error "max scheduling attempts reached 3"" [Critical,In progress] - Assigned to Steven Dake (sdake) | 16:11 |
| inc0 | not just for configuration of iscsi but actually to get it work | 16:12 |
| gmmaha | it has the details and can you test it in your setup | 16:12 |
| *** unicell has quit IRC | 16:14 | |
| sbezverk | inc0 I will reinstall one of my servers with ubuntu and troubleshoot it | 16:15 |
| inc0 | kk thanks | 16:15 |
| inc0 | let me know if I can help | 16:15 |
| inc0 | fyi version I use is 15.10, I would be ok if it works for 14.04 | 16:15 |
| inc0 | but that being said, 16.04 is just in front of us | 16:16 |
| sbezverk | inc0 sure, you were going to think about conditional volumes, any progress there? | 16:16 |
| *** banix has joined #openstack-kolla | 16:16 | |
| inc0 | sbezverk, didn't have time to work on it | 16:16 |
| inc0 | sorry | 16:16 |
| inc0 | I have ideas but need to test it | 16:16 |
| sbezverk | inc0 ok, let me know if you come with some ideas to try. | 16:16 |
| inc0 | sure | 16:17 |
| ccesario | vhosakot, pong | 16:23 |
| ccesario | vhosakot, my kolla-ansible is untouched CONFIG_DIR point to /etc/kolla | 16:23 |
| vhosakot | so, what is the issue then :) | 16:24 |
| vhosakot | ccesario: if CONFIG_DIR is changed ? | 16:24 |
| sbezverk | inc0 can you try to add this line to /etc/fstab and retry:configfs /sys/kernel/config configfs defaults 0 0 | 16:25 |
| ccesario | kolla-ansible --config-dir mydir | 16:25 |
| sbezverk | inc0 some people were reporting this problem but just with regular iscsi on ubuntu | 16:25 |
| inc0 | sbezverk, do I need to reboot? | 16:27 |
| *** Marga_ has joined #openstack-kolla | 16:27 | |
| sbezverk | inc0 you can mount it manually | 16:28 |
| inc0 | tried, not working, I'm reading that there is some modules I need to enable | 16:29 |
| inc0 | can't do now tho | 16:30 |
| sbezverk | inc0 let me know if you mae some progress | 16:31 |
| inc0 | sure | 16:31 |
| sbezverk | before I bring new server | 16:31 |
| inc0 | gtg now | 16:31 |
| *** barra204 has quit IRC | 16:32 | |
| *** barra204 has joined #openstack-kolla | 16:32 | |
| *** inc0 has quit IRC | 16:36 | |
| vhosakot | ccesario: yes, "kolla-ansible --configdir <directory>" is a valid use case.. I will fix it.. | 16:36 |
| ccesario | vhosakot, this it seem solve the problem.... http://pastebin.com/19i3F0hD | 16:36 |
| *** Marga_ has quit IRC | 16:36 | |
| ccesario | seems* | 16:36 |
| *** barra204 has quit IRC | 16:36 | |
| vhosakot | yep, plus I also need to exclude nova ssh key | 16:37 |
| ccesario | vhosakot, right | 16:37 |
| vhosakot | will submit the PS soon... I'm testing it no | 16:37 |
| vhosakot | now* | 16:37 |
| ccesario | vhosakot, nice | 16:37 |
| vhosakot | cool | 16:38 |
| ccesario | vhosakot, why not usage failed_when: result.rc != "1" instead failed_when: result.stdout != "" ? | 16:38 |
| sbezverk | have you noticed that the base image is failing to build? | 16:38 |
| sbezverk | http://paste.openstack.org/show/492899/ | 16:39 |
| vhosakot | ccesario: I will add you as co-author | 16:43 |
| *** banix has quit IRC | 16:43 | |
| mlima | hello guys, someone can review it https://review.openstack.org/#/c/299534/ it ended up being forgotten and it's a simple change. | 16:43 |
| patchbot | mlima: patch 299534 - kolla - Fix manila keystone v3 issues for all clients | 16:43 |
| *** inc0 has joined #openstack-kolla | 16:43 | |
| *** salv-orlando has joined #openstack-kolla | 16:43 | |
| *** inc0 has quit IRC | 16:44 | |
| *** Marga_ has joined #openstack-kolla | 16:44 | |
| *** Serlex has joined #openstack-kolla | 16:44 | |
| sdake | mlima see response in bug log for your change | 16:47 |
| mlima | sdake, i'll check ok? | 16:48 |
| sdake | sounds good | 16:48 |
| ccesario | vhosakot, don worry man :) the important is it works as expected | 16:52 |
| *** salv-orlando has quit IRC | 16:52 | |
| *** Serlex has quit IRC | 16:53 | |
| mlima | sdake, i think that mitaka was affected, the patchset was merged 2 weeks ago | 16:53 |
| sdake | mlima tht iss speculation, id' like to see the wrnings (or you see them) in mitaka | 16:54 |
| sdake | with stable/mitaka | 16:54 |
| mlima | yes | 16:54 |
| mlima | i'm checking it now :) | 16:54 |
| sdake | thanks ;) | 16:54 |
| sdake | the reason is i dont want to break mitaka | 16:54 |
| sdake | since our gate is snafu | 16:54 |
| sdake | because rdorpoject.org is dead | 16:54 |
| *** banix has joined #openstack-kolla | 16:55 | |
| sdake | looks like rdoproject..org is back alive ;) | 16:57 |
| mlima | sdake, the milestone in "original" bug is Manila mitaka-rc1 "RC1" | 16:58 |
| sdake | whole lot of bugs not fixed here: https://launchpad.net/kolla/+milestone/2.0.0 | 16:58 |
| sdake | if folks looking for things to do, please please work on those bugs ;) | 16:58 |
| mlima | but i want to confirm it myself :) | 16:58 |
| sdake | if your already working on a bug continue doing that of course ;) | 16:59 |
| ccesario | sdake, https://bugs.launchpad.net/bugs/1520728 already it was not solved in ps https://review.openstack.org/300655 ? | 17:01 |
| openstack | Launchpad bug 1520728 in kolla mitaka "fatal: [openstack002] => One or more undefined variables: 'dict object' has no attribute 'stdout'" [Critical,Confirmed] | 17:01 |
| sdake | ccesario i hae no idea | 17:02 |
| sdake | that is why i brought it into the release | 17:02 |
| sdake | so it could be verified | 17:02 |
| sdake | i was hoping someone that had that bug would try again with stable/mitaka | 17:02 |
| sdake | altnerately someone would verify it | 17:03 |
| *** rmart04 has quit IRC | 17:03 | |
| sdake | if its a dupe it would be good to know ;) | 17:03 |
| sdake | because syaing we wont fix things because of ansible bugs is totally not acceptable | 17:03 |
| sdake | when the bug affects a majority of deployments | 17:03 |
| sdake | rhallisey ping | 17:04 |
| *** jasonsb has quit IRC | 17:04 | |
| rhallisey | sdake, hey | 17:04 |
| ccesario | I 'm asking it... because the errors (deploy logs) it the same as https://bugs.launchpad.net/kolla/+bug/1546789 | 17:06 |
| openstack | Launchpad bug 1546789 in kolla mitaka "neutron failed to deploy in a multi-node deployment" [Critical,Fix committed] - Assigned to Steven Dake (sdake) | 17:06 |
| ccesario | and it was 'fixed' on ps https://review.openstack.org/30065 | 17:06 |
| *** rajathagasthya has joined #openstack-kolla | 17:07 | |
| sdake | yes but that doene't mean its the same bug ;) | 17:07 |
| *** unicell has joined #openstack-kolla | 17:08 | |
| sdake | but if yo uwant to focus on somethin g else then verifying a dupe that might be more productive :) | 17:08 |
| ccesario | yes, I agree | 17:08 |
| *** prithiv has quit IRC | 17:09 | |
| andrewvwebber | has anyone investigated kolla on coreos (no kubernetes) | 17:09 |
| *** prithiv has joined #openstack-kolla | 17:09 | |
| *** allen_gao has quit IRC | 17:12 | |
| *** allen_gao has joined #openstack-kolla | 17:13 | |
| *** prithiv has quit IRC | 17:13 | |
| *** unicell has joined #openstack-kolla | 17:14 | |
| mlima | sdake, mitaka was affected, buuuuuuuuuut i'll check if it still work even with warnings | 17:16 |
| *** andrewvwebber has quit IRC | 17:17 | |
| mlima | if still work, I see no reason to merge it in mitaka | 17:17 |
| sdake | mlima i'd like to backport what works and is a legitimate bug fix | 17:25 |
| sdake | so if it works lets backport it | 17:26 |
| sdake | give me confirmation and i'll make the rest happen | 17:26 |
| mlima | your wish is an order | 17:27 |
| mlima | you have two more wish | 17:28 |
| sdake | mlima i dont make orders bro | 17:31 |
| sdake | you do what y ou want ;) | 17:31 |
| mlima | :) | 17:31 |
| mandre | sdake: here now | 17:34 |
| mandre | what's up? | 17:34 |
| sdake | hey folks | 17:36 |
| sdake | I wanted to let everyone know | 17:36 |
| sdake | we ended up with 4 fishbowl sessions | 17:36 |
| sdake | and 10 design sessions | 17:36 |
| sdake | out of the 24 we asked for | 17:36 |
| sdake | the 14 is a serous haul | 17:36 |
| sdake | very few projects got that much agenda time | 17:37 |
| sdake | also we have a full day contributor meetup friday | 17:37 |
| sdake | so try to stay until saturday if possible | 17:37 |
| sdake | i'd like to go over a gate code review so folks understand how the gate works on friday | 17:37 |
| mandre | nice | 17:37 |
| sdake | and finish our ta work | 17:37 |
| *** SiRiuS has quit IRC | 17:39 | |
| *** vhosakot has quit IRC | 17:39 | |
| mandre | hello from France BTW :) | 17:39 |
| *** SiRiuS has joined #openstack-kolla | 17:42 | |
| *** vhosakot has joined #openstack-kolla | 17:43 | |
| *** coolsvap has quit IRC | 17:44 | |
| *** dave-mccowan has joined #openstack-kolla | 17:50 | |
| rhallisey | mandre, oh hai | 17:51 |
| rhallisey | mandre, did you have a nice move? | 17:51 |
| mandre | hey rhallisey, nice to see you at normal time | 17:51 |
| rhallisey | I know! | 17:51 |
| mandre | yeah, all went as smooth as it could | 17:52 |
| mandre | minus the strikes in France :-/ | 17:52 |
| *** coolsvap has joined #openstack-kolla | 17:56 | |
| sdake | vhosakot welcome to the ore revier team | 18:01 |
| sdake | i have a meeting right now with a customer | 18:01 |
| sdake | i wil provide instructions in 45 mins to 2 hours | 18:01 |
| sdake | an dmake the privilege changess | 18:01 |
| sdake | france always srikes | 18:02 |
| sdake | ;) | 18:02 |
| *** vhosakot has quit IRC | 18:02 | |
| gmmaha | hello everyone.. quick Q.. if i want some config data injected into nova.conf, would adding them to /etc/kolla/config/nova.conf work? | 18:10 |
| *** salv-orlando has joined #openstack-kolla | 18:11 | |
| gmmaha | i think vhosakot mentioned about it.. trying to avoid fiddling with the nova.conf.j2 file | 18:11 |
| *** gfidente has quit IRC | 18:12 | |
| *** banix has quit IRC | 18:15 | |
| rhallisey | gmmaha, yes. Each service's config file gets copied in | 18:15 |
| rhallisey | if you have containers running be sure to be using 'COPY_ALWAYS' | 18:16 |
| gmmaha | rhallisey: when you say copied in, Its unification of wht we get from nova.conf.j2 and the contents of /etc/kolla/config/nova.conf right? | 18:16 |
| gmmaha | rhallisey: aah i detroyed all the containers.. so its a fresh deploy | 18:16 |
| *** daneyon_ has joined #openstack-kolla | 18:18 | |
| rhallisey | gmmaha, should see a unification | 18:19 |
| gmmaha | rhallisey: awesome.. thank you | 18:20 |
| *** daneyon has quit IRC | 18:20 | |
| *** mgoddard_ has joined #openstack-kolla | 18:23 | |
| *** mbound has quit IRC | 18:25 | |
| *** mgoddard has quit IRC | 18:27 | |
| *** salv-orlando has quit IRC | 18:27 | |
| *** joliver has quit IRC | 18:27 | |
| *** mgoddard_ has quit IRC | 18:28 | |
| *** banix has joined #openstack-kolla | 18:29 | |
| *** salv-orlando has joined #openstack-kolla | 18:32 | |
| *** diogogmt has joined #openstack-kolla | 18:34 | |
| *** mgoddard has joined #openstack-kolla | 18:37 | |
| *** salv-orlando has quit IRC | 18:43 | |
| *** vhosakot has joined #openstack-kolla | 19:00 | |
| gmmaha | rhallisey: have you ever seen this error before? | 19:02 |
| gmmaha | http://paste.openstack.org/show/492911/ | 19:02 |
| rhallisey | gmmaha, are you using a local docker registry? | 19:02 |
| gmmaha | rhallisey: yes.. | 19:03 |
| rhallisey | gmmaha, the image '172.16.2.2:4000/kollaglue/ubuntu-source-heka:2.0.0' is missing | 19:03 |
| rhallisey | or that node is unable to connect to your registry | 19:03 |
| gmmaha | rhallisey: i checked.. the image is there.. | 19:03 |
| gmmaha | let me kill the registry and bring it back up again | 19:04 |
| rhallisey | gmmaha, on the node that is looking for the registry | 19:04 |
| rhallisey | $ systemctl status docker | 19:04 |
| rhallisey | see if it | 19:04 |
| rhallisey | it | 19:04 |
| rhallisey | see if its running with --insecure-registry 172.16.2.2:4000' | 19:04 |
| gmmaha | rhallisey: aah its running like this. /usr/bin/docker daemon --insecure-registry 172.16.2.241:4000 | 19:04 |
| gmmaha | wrong IP :) | 19:05 |
| rhallisey | gmmaha, there you go :) | 19:05 |
| gmmaha | interesting.. all my mahcines are running like that.. | 19:05 |
| *** Marga_ has quit IRC | 19:06 | |
| *** Marga_ has joined #openstack-kolla | 19:06 | |
| rhallisey | gmmaha, https://github.com/openstack/kolla/blob/master/etc/kolla/globals.yml#L49 | 19:09 |
| *** salv-orlando has joined #openstack-kolla | 19:09 | |
| gmmaha | rhallisey: thats set to the right value.. | 19:10 |
| gmmaha | i just fixed /etc/default/docker on all the nodes and restarted docker | 19:10 |
| gmmaha | let me try and deploy again | 19:10 |
| rhallisey | gmmaha, see if you can $ docker pull 172.16.2.2:4000/kollaglue/ubuntu-source-heka:2.0.0 | 19:10 |
| gmmaha | rhallisey: aah yes let me check that | 19:11 |
| gmmaha | i was trigger happy and started the deploy again and its working now.. I see the heka and kolla-toolbox containers being deployed | 19:11 |
| gmmaha | :) | 19:11 |
| gmmaha | thanks a lot rhallisey | 19:11 |
| rhallisey | gmmaha, nice! | 19:11 |
| *** sdake_ has joined #openstack-kolla | 19:12 | |
| *** Marga_ has quit IRC | 19:12 | |
| *** mgoddard has quit IRC | 19:14 | |
| *** salv-orlando has quit IRC | 19:14 | |
| *** salv-orlando has joined #openstack-kolla | 19:14 | |
| *** sdake has quit IRC | 19:15 | |
| *** salv-orlando has quit IRC | 19:16 | |
| *** salv-orlando has joined #openstack-kolla | 19:16 | |
| *** salv-orlando has quit IRC | 19:17 | |
| *** salv-orlando has joined #openstack-kolla | 19:17 | |
| *** daneyon has joined #openstack-kolla | 19:18 | |
| *** mlima_ has joined #openstack-kolla | 19:19 | |
| *** daneyon_ has quit IRC | 19:20 | |
| *** mlima has quit IRC | 19:23 | |
| *** mbound has joined #openstack-kolla | 19:25 | |
| *** mbound has quit IRC | 19:30 | |
| *** inc0 has joined #openstack-kolla | 19:41 | |
| *** shakamunyi has joined #openstack-kolla | 19:51 | |
| g3ek | hello guys, quick question. I just saw that I can't use a variable like ansible_eth2 from another group if I not running the group in the deploy.yml. Exemple : include: bootstrap.yml \n when: inventory_hostname != groups['ormuco'][0] If I do that, I cannot get the ansible_eth2 from groups['nodes'] from my template. All interface are not set for those server. Is their a way to force the detecting of the network config to all server before runnin | 19:54 |
| g3ek | g any template. | 19:54 |
| *** banix has quit IRC | 19:57 | |
| *** shakamunyi has quit IRC | 19:57 | |
| *** dwalsh has quit IRC | 19:57 | |
| *** banix has joined #openstack-kolla | 19:58 | |
| manjeets | sdake today i saw machine getting 5 ips | 19:58 |
| manjeets | that was boot right after deploy | 19:59 |
| openstackgerrit | Mauricio Lima proposed openstack/kolla: Add nova_ssh in cleanup containers script https://review.openstack.org/301319 | 20:03 |
| sdake_ | manjeets tht is a separae problem | 20:05 |
| sdake_ | manjeets cn you launch 100 vms now with my patch applied | 20:05 |
| sdake_ | manjeets i will get your othe problem fixed | 20:06 |
| sdake_ | but one at a time | 20:06 |
| sdake_ | i got the database one fixed, right? :-) | 20:06 |
| sdake_ | manjeets the patchto cherrypick itno your repo is here https://bugs.launchpad.net/bugs/1563664 | 20:09 |
| openstack | Launchpad bug 1563664 in kolla "creating 100 vms via horizon on 64 node cluster returns error "max scheduling attempts reached 3"" [Critical,In progress] - Assigned to Steven Dake (sdake) | 20:09 |
| manjeets | i did not do 50 | 20:09 |
| manjeets | i said 10 where you had 50 in your patch | 20:09 |
| sdake_ | and that worked? | 20:10 |
| sdake_ | consistently? | 20:10 |
| manjeets | sdake_ i am pretty sure kolla is probelmatic | 20:10 |
| sdake_ | its config options only not kolla | 20:10 |
| manjeets | it does so on initial runs after deployment | 20:10 |
| sdake_ | ok here is what happens after an initial deploy | 20:10 |
| sdake_ | you say "spin up 100 vms" | 20:10 |
| sdake_ | the nova schedule attempts to shcedule that | 20:11 |
| manjeets | on same deployment things start getting nice after 2 3 runs of scheduling | 20:11 |
| sdake_ | it pulls the cirros imae | 20:11 |
| sdake_ | that cirros imae pulling takes time | 20:11 |
| manjeets | now 100 on same deployment got active so fast | 20:11 |
| sdake_ | since the cirros imae pulling takes time, the scheduling times out | 20:11 |
| sdake_ | and retries | 20:11 |
| sdake_ | that is why i cranked up the retry count | 20:12 |
| sdake_ | for the startup state of scheduling | 20:12 |
| sdake_ | let me ask thiis | 20:12 |
| sdake_ | did you receive inthe ogs "max scheduign attempts reached X" | 20:12 |
| sdake_ | where X is whtever number you used | 20:12 |
| sdake_ | once the cirros imae is pulled into the local cache | 20:13 |
| sdake_ | the scheduler doesn't time ut anymore | 20:13 |
| sdake_ | thsi is why the 2nd time you try it works | 20:13 |
| sdake_ | or maybe the third time because the 2nd time some images are stil pulling | 20:14 |
| *** sdake_ is now known as sdake | 20:14 | |
| manjeets | not this time when i raise it to 10 | 20:15 |
| *** Marga_ has joined #openstack-kolla | 20:15 | |
| *** Marga_ has quit IRC | 20:16 | |
| manjeets | but port creation and ip allocation is serious issue that happens when you boot vms after deploy | 20:16 |
| *** mlima_ has quit IRC | 20:17 | |
| sdake | that is a different bug | 20:17 |
| sdake | one thing at time dude :) | 20:17 |
| manjeets | i did not max retries when i raised it to 10 | 20:17 |
| manjeets | for 100 | 20:17 |
| sdake | can you confirm yu no longer receive th eerror max scheduling attempts seen | 20:18 |
| sdake | huh? | 20:18 |
| sdake | you didn't get that error you mean? | 20:18 |
| manjeets | i mean i don't see max tries reached upto 100 now when i raised that limit to 10 | 20:18 |
| sdake | nice | 20:18 |
| manjeets | previously i saw on 100 some times as well | 20:18 |
| sdake | so comment in the review the problem no longer exists with that patch | 20:19 |
| manjeets | i commented on your patch | 20:19 |
| manjeets | 50 is way too much | 20:19 |
| sdake | oh didn't see tht in my inbox | 20:19 |
| manjeets | https://review.openstack.org/#/c/300914/1 | 20:20 |
| patchbot | manjeets: patch 300914 - kolla - Increase max scheduling attempts to 50 | 20:20 |
| sdake | https://bugs.launchpad.net/kolla/+bug/1565105 | 20:20 |
| openstack | Launchpad bug 1565105 in kolla mitaka "while spining vms on large scale some vms get multiple ips and multiple ports get created" [Critical,Confirmed] - Assigned to Steven Dake (sdake) | 20:20 |
| manjeets | i commented on sdake 's patch already | 20:20 |
| sdake | so 3 is too small | 20:20 |
| sdake | 50 is too big, why? | 20:20 |
| sdake | trying to pull up the review | 20:21 |
| sdake | my internets have been going slow lately | 20:21 |
| manjeets | 50 is way too much i think | 20:21 |
| sdake | so manjeet you said 50 is too high but you didn't say 10 worked | 20:21 |
| manjeets | ideally it should not do more than 5 tries | 20:22 |
| sdake | idelaly it shouldn't need to e rescheduled at allb | 20:22 |
| manjeets | i said i did not have problem when i raised it to 10 | 20:22 |
| sdake | but nova and glance interactions are defective | 20:22 |
| sdake | not in the review | 20:22 |
| sdake | say it in the review | 20:22 |
| sdake | that way the crs know its been tested | 20:23 |
| manjeets | ok | 20:23 |
| manjeets | sdake: so manjeet you said 50 is too high but you didn't say 10 worked | 20:23 |
| manjeets | it confused me because above message is missing on review | 20:23 |
| sdake | don't you think 50 is way too large i would say 10 is enough. | 20:24 |
| sdake | this is what you elft in ther review | 20:24 |
| *** dwalsh has joined #openstack-kolla | 20:24 | |
| manjeets | sdake will add more details | 20:24 |
| manjeets | to review on gerrit | 20:24 |
| sdake | thanks | 20:25 |
| sdake | in mean time i'll try cracking your other ip issue | 20:25 |
| sdake | i'd like you to try 1000 | 20:25 |
| sdake | with 10 | 20:25 |
| sdake | if that works | 20:25 |
| sdake | we can go with 10 | 20:25 |
| sdake | i htink the number 10 is dependent on number of compute nodes and number of vms scheduled at one time | 20:25 |
| *** dwalsh has quit IRC | 20:31 | |
| gmmaha | Hi, this seems to be a new error.. http://paste.openstack.org/show/492925/ | 20:32 |
| gmmaha | was this something that got added recently in the source tree | 20:32 |
| rhallisey | gmmaha, did you run /tools/generate_passwords? | 20:34 |
| *** sdake_ has joined #openstack-kolla | 20:34 | |
| *** sdake has quit IRC | 20:34 | |
| gmmaha | rhallisey: apparently i didnt.. not after updating the new tree | 20:34 |
| gmmaha | i pulled the latest and just assumed that nothign has been added.. my bad. | 20:34 |
| gmmaha | sorry | 20:34 |
| rhallisey | gmmaha, no worries | 20:34 |
| rhallisey | there might not be anything in the quickstart.. | 20:35 |
| gmmaha | rhallisey: i will check and push a doc update if i dont see it | 20:36 |
| rhallisey | gmmaha, just checked there isn't | 20:36 |
| rhallisey | there's a patch to add it as a precheck | 20:36 |
| gmmaha | rhallisey: aaah neat! | 20:37 |
| gmmaha | thanks for checking | 20:37 |
| inc0 | sbezverk, got conditional volumes working:) | 20:37 |
| inc0 | http://paste.openstack.org/show/492927/ | 20:37 |
| inc0 | mind refactoring your code to default disable iscsi? | 20:38 |
| sbezverk | inc0 nice!! let me check.. | 20:39 |
| manjeets | sdake i did not do any thing other than raising limit now 100 vms are getting active in like 15 seconds | 20:40 |
| inc0 | sbezverk, vhosakot will you guys attend summit? I want to have a beer with you;) | 20:45 |
| *** dwalsh has joined #openstack-kolla | 20:48 | |
| openstackgerrit | Merged openstack/kolla: Add comments on site.yml neutron hosts reordering https://review.openstack.org/300651 | 20:50 |
| vhosakot | inc0: yep, sure.. I will be at the summit! :) | 20:50 |
| inc0 | good to hear:)...or read | 20:51 |
| sdake_ | vhosakot welcome to the cr team :) | 20:53 |
| sdake_ | vhosakot couple pieces of advice | 20:53 |
| sdake_ | use -2 sparingly | 20:53 |
| vhosakot | sdake_: thanks! :) | 20:53 |
| vhosakot | yep.. | 20:53 |
| *** shakamunyi has joined #openstack-kolla | 20:53 | |
| vhosakot | thanks to the entire community! :) | 20:53 |
| sdake_ | if your just not sure on a patch but it looks good, + 1is ok | 20:53 |
| sdake_ | if your sure +2 is good | 20:53 |
| vhosakot | ok cool | 20:53 |
| sdake_ | ideally yo ushould be +2 or -1 most of he time on patches | 20:53 |
| vhosakot | ok cool | 20:54 |
| sdake_ | the idea is to get the -1 patches into a state where they can be +2ed | 20:54 |
| sdake_ | it takes 2 +2s before applying workflow | 20:54 |
| sdake_ | the econd person to review applie workflow +1 | 20:54 |
| sdake_ | if the gat elooks good | 20:54 |
| sdake_ | a workflow=1 will merge in th repo | 20:55 |
| sdake_ | even if there aren't two +2s | 20:55 |
| sdake_ | so be careful not to accidentally click | 20:55 |
| sdake_ | we dont make exceptions to our +2 policy | 20:55 |
| sdake_ | if your an author or coauthor you may not =2 a patch | 20:55 |
| sdake_ | unless its a cherrypick backport | 20:56 |
| sdake_ | and even if your the author you may not +2 | 20:56 |
| sdake_ | but if you cherrypicked it a +2 i sfine | 20:56 |
| sdake_ | as long as it didn't have collsisions | 20:56 |
| sdake_ | I rarely +1, only when i am not 100% confident in my understanding of the patch | 20:57 |
| sdake_ | but think it looks good | 20:57 |
| openstackgerrit | Carlos Cesario proposed openstack/kolla: Added hostname param into kolla_docker https://review.openstack.org/297984 | 20:57 |
| sdake_ | or want other crs to look at it | 20:57 |
| vhosakot | ok... cool | 20:57 |
| sdake_ | sometime you will be asked to vote on the mailing list | 20:58 |
| *** shakamunyi has quit IRC | 20:58 | |
| sdake_ | our votes equire quorum to pass | 20:58 |
| sdake_ | there is no veto | 20:58 |
| sdake_ | the only veto is on core nominations | 20:58 |
| vhosakot | yep, I will be careful about the "workflow" button :) | 20:58 |
| vhosakot | ah ok | 20:58 |
| sdake_ | votes re open for 1 week typically | 20:58 |
| vhosakot | I will vote accordingly in ML as well.. cool | 20:58 |
| vhosakot | ok | 20:58 |
| sdake_ | i use voting when i dont want to make a decision in isolation but want to follow the will of the community | 20:59 |
| sdake_ | or there is conflict between the cores how to proceed | 20:59 |
| sdake_ | since we have 12 crs now | 20:59 |
| sdake_ | 7 = quorum | 20:59 |
| sdake_ | any q&a, not just from vikram, but from anyone? | 21:00 |
| vhosakot | ah, right... 7 is majority | 21:01 |
| sdake_ | crs should make an attempt to attend our weekly irc meeting unless its for example 3am | 21:02 |
| sdake_ | in your cse, i think yu can make both meeting slots | 21:02 |
| sdake_ | there is no harm in stepping away from work you have signedu p for | 21:02 |
| sdake_ | just unassign yourself | 21:02 |
| sdake_ | so others know it i svailable | 21:02 |
| sdake_ | no harm no shame | 21:02 |
| sdake_ | sometimes capacity changes that we think we hd and don't | 21:03 |
| sdake_ | if it becomes consistent, that coud become a problem ;) | 21:03 |
| sdake_ | ok i nee dto catch a powe rnap | 21:03 |
| sdake_ | bbl | 21:03 |
| sdake_ | manjeets i will file a review for your ip problem when I'm a little fresher | 21:03 |
| vhosakot | yep, I always attend meeting.. cool | 21:05 |
| manjeets | sdake_ i already have a launcpad ticket for that | 21:06 |
| *** ccesario_ has joined #openstack-kolla | 21:07 | |
| *** ccesario___ has joined #openstack-kolla | 21:08 | |
| *** ccesario_ has quit IRC | 21:08 | |
| *** mgoddard has joined #openstack-kolla | 21:08 | |
| *** rajathagasthya has quit IRC | 21:13 | |
| *** sdake_ has quit IRC | 21:14 | |
| *** banix has quit IRC | 21:16 | |
| *** ccesario___ has quit IRC | 21:30 | |
| *** ccesario_ has joined #openstack-kolla | 21:31 | |
| *** ccesario___ has joined #openstack-kolla | 21:33 | |
| g3ek | sdake: Can you explain me why all kolla-nodes variables are not initialise directly at the kolla-ansible deploy start? I have a problem right now with my new module. I need my module to run directly before the keystone module. I need to access some kolla-nodes (like the compute eth2 IP), but it's not working because no action have been taken on the compute node yet and the ansible_eth2 is not declared. | 21:34 |
| g3ek | sdake: if I put my module at the end of the role.yml file, everything work | 21:34 |
| *** ccesario_ has quit IRC | 21:37 | |
| *** dwalsh has quit IRC | 21:45 | |
| *** mgoddard has quit IRC | 21:46 | |
| g3ek | can someone read my last question to sdake and answer me ? I need to understand | 21:47 |
| *** haplo37 has quit IRC | 21:51 | |
| *** sdake has joined #openstack-kolla | 21:54 | |
| vhosakot | g3ek: can you pastebin the error... I don't see ansible_eth2 in the code... sounds like a hostvar | 21:58 |
| *** inc0 has quit IRC | 22:07 | |
| g3ek | vhosakot : This is a example. I added the value in groups_var/all.yml : It's the same with eth0 and eth1 | 22:08 |
| vhosakot | g3ek: you added them in globals.yml as well | 22:10 |
| vhosakot | ? | 22:10 |
| *** mwheckmann has joined #openstack-kolla | 22:14 | |
| g3ek | like if I want to do this : http://pastebin.com/udntsuCR | 22:15 |
| g3ek | vhosakot nope, not in the globals.yml | 22:16 |
| g3ek | but like I said if I run my module in the last place, It work perfectly. The problem is that I need it to run direct after the heka part | 22:17 |
| rhallisey | g3ek, https://github.com/openstack/kolla/tree/master/ansible/roles/common/tasks | 22:19 |
| g3ek | I tought that kolla will run the globals.yml on all nodes before starting the deployment. It seems to be not like that | 22:20 |
| g3ek | It run the globals.yml on the node before doing the deployment on that node | 22:20 |
| rhallisey | g3ek, https://github.com/openstack/kolla/blob/master/ansible/roles/common/tasks/start.yml | 22:20 |
| rhallisey | right in there is what your looking for I think | 22:21 |
| g3ek | Nope, I need something that will run the globals.yml on the compute before doing my DNS. The eth2 is not declare on the compute yet, because nothing have been installed on it yet... | 22:22 |
| *** rajathagasthya has joined #openstack-kolla | 22:22 | |
| g3ek | rhallisey: sorry forget to tag you | 22:22 |
| g3ek | if I run the ansible -m setup -a 'filter=ansible_eth2' on the nodes group it work perfectly. | 22:23 |
| g3ek | A solution would be to create a fake module that will run nothing on the compute and then run my DNS module, but this is not very clean ... | 22:25 |
| g3ek | just to initalize all the vars on it | 22:25 |
| rhallisey | g3ek, so you have stuff on globals that isn't getting to your nodes.. | 22:28 |
| rhallisey | when you need it | 22:28 |
| g3ek | yes, just if no action have been done on that node | 22:28 |
| *** RRRR1223 has joined #openstack-kolla | 22:28 | |
| g3ek | if heka pass on the node, it will work perfectly | 22:28 |
| g3ek | just because the hostvars[host] will have all globals into it | 22:29 |
| *** RRRR1223 has quit IRC | 22:29 | |
| rhallisey | g3ek, what happens if you add your module here: https://github.com/openstack/kolla/blob/master/ansible/inventory/all-in-one#L18 | 22:30 |
| g3ek | rhallisey: already there | 22:31 |
| g3ek | rhallisey: I thing this is a design bug | 22:31 |
| g3ek | rhallisey: Like if it was not planned to get some informations from another groups/host when you are not doing action into it. | 22:32 |
| rhallisey | were just not setting this up right | 22:34 |
| *** mwheckmann has quit IRC | 22:34 | |
| *** ccesario___ has quit IRC | 22:34 | |
| *** ccesario___ has joined #openstack-kolla | 22:35 | |
| g3ek | https://objects.ormuco.com/swift/v1/Kolla/Controller-1.png | 22:35 |
| g3ek | https://objects.ormuco.com/swift/v1/Kolla/compute-1.png | 22:35 |
| g3ek | check this | 22:35 |
| g3ek | this is the difference betwwen a initialized node and a no-initialize node | 22:35 |
| sdake | ok time to go desk shopping | 22:35 |
| sdake | bbl | 22:35 |
| *** sdake has quit IRC | 22:36 | |
| g3ek | their is no information about ETH card | 22:36 |
| g3ek | rhallisey: forget the tag again | 22:37 |
| rhallisey | what did you run to get this output | 22:37 |
| g3ek | I run the template, but instead of getting the ansible_eth2 value a get the hostvar[host] | 22:38 |
| rhallisey | I mean in the picture | 22:38 |
| rhallisey | is there an ansible command | 22:38 |
| g3ek | {% for host in groups['nodes'] %} | 22:38 |
| g3ek | {{ host }}.{{ ormuco_zone_domain_name }}. IN A {{ hostvars[host] }} | 22:38 |
| g3ek | {% endfor %} | 22:38 |
| rhallisey | to give you all the vars set on a node | 22:38 |
| g3ek | yes | 22:38 |
| g3ek | and their is a long diffence | 22:38 |
| g3ek | It's not an ansible command, its an ansible variable that will store all vars from a host called hostvars[NODENAME] | 22:40 |
| g3ek | rhallisey | 22:40 |
| rhallisey | ya gotcha | 22:41 |
| rhallisey | I was hoping it was an ansible command just for future reference | 22:41 |
| g3ek | Do you understand better my problem now ? | 22:41 |
| g3ek | you can do it with ansible -m setup without -a 'filter=SOMETHING' | 22:42 |
| g3ek | I think Kolla would need to initialize all those vars first before starting the deployment. It will prevent that to happen. | 22:44 |
| g3ek | rhallisey | 22:44 |
| rhallisey | g3ek, so the stuff you added in globals shows up after the first container spawns? | 22:44 |
| rhallisey | or after heka | 22:44 |
| g3ek | yes, but not the ansible_eth2 eth1 and eth0 var | 22:45 |
| g3ek | ohhh, if i run hecka and then run my module it work for both controller | 22:45 |
| g3ek | but since the compute and the storage have never been touched it raise an exeption | 22:45 |
| g3ek | on them | 22:46 |
| g3ek | like if I do : {% for host in groups['control'] %} | 22:47 |
| g3ek | {{ host }}.{{ ormuco_zone_domain_name }}. IN A {{ hostvars[host] }} | 22:47 |
| g3ek | {% endfor %} | 22:47 |
| g3ek | It work perfectly. All controller eth card are into the hostvars | 22:47 |
| g3ek | if after heka I run the cinder module and the nova-compute module (imagine it's possible). Now all hostvars from the compute and the storage will be define and everything will work | 22:49 |
| rhallisey | g3ek, ok so some of the vars make it ove | 22:49 |
| rhallisey | over | 22:49 |
| rhallisey | but not the ones you need | 22:49 |
| g3ek | once you pass on the node one time all vars a define | 22:49 |
| g3ek | do you have hangout? will be more asy to explain with share screen and speaking | 22:50 |
| rhallisey | g3ek, but when you're trying to run your play, your network_interface does not show up there yet | 22:50 |
| g3ek | not just mine, all others too. If I run some module on the controller like the heka module, it will define all the network interface on both controller since this module run on them. But heka is not installed on the compute and the storage, so for them all network interface are still unknowed. Like both screenshot I sent you show | 22:52 |
| *** Marga_ has joined #openstack-kolla | 22:54 | |
| g3ek | rhallisey | 22:55 |
| rhallisey | g3ek, heka should get put on all node I think | 22:56 |
| rhallisey | pretty sure it's a common play | 22:56 |
| *** diogogmt has quit IRC | 22:56 | |
| *** Marga_ has quit IRC | 22:56 | |
| g3ek | rhalliseym heka are started on the controller only. After that all other module use the heka volume to mount it to the module | 22:56 |
| *** Marga_ has joined #openstack-kolla | 22:56 | |
| *** Marga_ has quit IRC | 22:56 | |
| *** Marga_ has joined #openstack-kolla | 22:57 | |
| g3ek | rhallisey, it's common because all module use it, but not all nodes run a heka containers. Just the controllers. Other node just mount the volume from the heka's container | 22:58 |
| *** vhosakot has quit IRC | 22:58 | |
| *** diogogmt has joined #openstack-kolla | 22:59 | |
| rhallisey | just a sec | 23:01 |
| *** shakamunyi has joined #openstack-kolla | 23:03 | |
| *** shakamunyi has quit IRC | 23:16 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/kolla: Updated from global requirements https://review.openstack.org/301391 | 23:17 |
| *** salv-orlando has quit IRC | 23:21 | |
| *** shakamunyi has joined #openstack-kolla | 23:27 | |
| *** banix has joined #openstack-kolla | 23:28 | |
| *** shakamunyi has quit IRC | 23:31 | |
| sbezverk | anyone from core could take a look at this please: https://review.openstack.org/#/c/300904/ | 23:35 |
| patchbot | sbezverk: patch 300904 - kolla - Adds migration step from nova to nova_api database | 23:35 |
| manjeets | sdake inc0 anybody have any idea what is beam.smp process ? | 23:38 |
| manjeets | is it related to rabbitmq ? | 23:38 |
| ccesario | vhosakot ping ? | 23:42 |
| *** shakamunyi has joined #openstack-kolla | 23:43 | |
| *** shakamunyi has quit IRC | 23:47 | |
| *** SiRiuS has quit IRC | 23:53 | |
| *** rajathagasthya has quit IRC | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
