| *** speedmann has joined #openstack-ansible | 00:01 | |
| *** speedmann has quit IRC | 00:02 | |
| *** speedmann has joined #openstack-ansible | 00:05 | |
| *** speedmann has quit IRC | 00:05 | |
| *** speedmann has joined #openstack-ansible | 00:06 | |
| *** speedmann has quit IRC | 00:06 | |
| *** throwsb1 has quit IRC | 00:13 | |
| *** speedmann has joined #openstack-ansible | 00:15 | |
| *** speedmann has quit IRC | 00:15 | |
| *** jamesde__ has quit IRC | 00:15 | |
| *** jamesden_ has joined #openstack-ansible | 00:17 | |
| *** speedmann has joined #openstack-ansible | 00:22 | |
| *** gyee has quit IRC | 00:23 | |
| *** speedmann has quit IRC | 00:23 | |
| *** speedmann has joined #openstack-ansible | 00:28 | |
| *** speedmann has quit IRC | 00:29 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Restructure Journalbeat so it respects package_state=absent https://review.openstack.org/585135 | 00:29 |
|---|---|---|
| *** speedmann has joined #openstack-ansible | 00:31 | |
| *** speedmann has quit IRC | 00:31 | |
| *** jwitko_ has joined #openstack-ansible | 00:32 | |
| *** speedmann has joined #openstack-ansible | 00:33 | |
| *** speedmann has quit IRC | 00:33 | |
| *** jwitko has quit IRC | 00:35 | |
| *** speedmann has joined #openstack-ansible | 00:40 | |
| *** speedmann has quit IRC | 00:40 | |
| *** speedmann has joined #openstack-ansible | 00:41 | |
| *** speedmann has quit IRC | 00:41 | |
| *** speedmann has joined #openstack-ansible | 00:42 | |
| *** speedmann has quit IRC | 00:42 | |
| *** speedmann has joined #openstack-ansible | 00:44 | |
| *** mmercer has quit IRC | 01:57 | |
| *** harryhuang_ has quit IRC | 02:06 | |
| *** harryhuang_ has joined #openstack-ansible | 02:07 | |
| *** yuzhiqiang has joined #openstack-ansible | 02:27 | |
| *** harryhuang_ has quit IRC | 02:29 | |
| *** harryhuang_ has joined #openstack-ansible | 02:31 | |
| *** jonher has joined #openstack-ansible | 03:12 | |
| *** jonher_ has quit IRC | 03:14 | |
| *** dave-mccowan has quit IRC | 03:24 | |
| *** speedmann has quit IRC | 03:34 | |
| *** speedmann has joined #openstack-ansible | 03:36 | |
| *** speedmann has quit IRC | 03:36 | |
| *** speedmann has joined #openstack-ansible | 03:41 | |
| *** speedmann has quit IRC | 03:41 | |
| *** speedmann has joined #openstack-ansible | 03:43 | |
| *** speedmann has joined #openstack-ansible | 03:44 | |
| *** speedmann has quit IRC | 03:45 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Convert logstash groks to a multi-pipeline setup https://review.openstack.org/586419 | 03:48 |
| *** speedmann has joined #openstack-ansible | 03:51 | |
| *** speedmann has quit IRC | 03:51 | |
| *** poopcat has quit IRC | 04:00 | |
| *** speedmann has joined #openstack-ansible | 04:06 | |
| *** speedmann has quit IRC | 04:09 | |
| *** pcaruana has joined #openstack-ansible | 04:28 | |
| *** pcaruana has quit IRC | 04:30 | |
| *** speedmann has joined #openstack-ansible | 04:32 | |
| *** speedmann has quit IRC | 04:33 | |
| *** speedmann has joined #openstack-ansible | 04:40 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Convert logstash groks to a multi-pipeline setup https://review.openstack.org/586419 | 04:40 |
| *** speedmann has joined #openstack-ansible | 04:42 | |
| *** shyamb has joined #openstack-ansible | 04:46 | |
| *** speedmann has joined #openstack-ansible | 04:47 | |
| *** speedmann has quit IRC | 04:47 | |
| *** speedmann has joined #openstack-ansible | 04:49 | |
| *** speedmann has quit IRC | 04:50 | |
| *** speedmann has joined #openstack-ansible | 04:50 | |
| *** speedmann has joined #openstack-ansible | 05:05 | |
| *** speedmann has quit IRC | 05:06 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Convert logstash groks to a multi-pipeline setup https://review.openstack.org/586419 | 05:10 |
| *** speedmann has joined #openstack-ansible | 05:13 | |
| *** Bhujay has joined #openstack-ansible | 05:17 | |
| *** Bhujay has quit IRC | 05:21 | |
| *** shyamb has quit IRC | 06:15 | |
| *** shyamb has joined #openstack-ansible | 06:15 | |
| *** lvdombrkr has joined #openstack-ansible | 06:18 | |
| *** shyamb has quit IRC | 06:22 | |
| *** cshen has joined #openstack-ansible | 06:26 | |
| *** cshen has quit IRC | 06:29 | |
| *** lvdombrkr89 has joined #openstack-ansible | 06:31 | |
| *** cshen has joined #openstack-ansible | 06:32 | |
| *** lvdombrkr has quit IRC | 06:33 | |
| *** shyamb has joined #openstack-ansible | 06:35 | |
| *** yuzhiqiang has quit IRC | 06:35 | |
| *** shyambiradar has joined #openstack-ansible | 06:44 | |
| *** shyamb has quit IRC | 06:47 | |
| *** speedmann has quit IRC | 06:58 | |
| *** speedmann has joined #openstack-ansible | 06:58 | |
| *** speedmann has quit IRC | 06:59 | |
| *** speedmann has joined #openstack-ansible | 07:01 | |
| *** speedmann has quit IRC | 07:01 | |
| *** speedmann has joined #openstack-ansible | 07:03 | |
| *** speedmann has quit IRC | 07:03 | |
| *** speedmann has joined #openstack-ansible | 07:11 | |
| *** speedmann has quit IRC | 07:11 | |
| *** speedmann has joined #openstack-ansible | 07:13 | |
| *** speedmann has quit IRC | 07:14 | |
| *** speedmann has joined #openstack-ansible | 07:15 | |
| *** speedmann has quit IRC | 07:15 | |
| *** speedmann has joined #openstack-ansible | 07:16 | |
| *** speedmann has quit IRC | 07:16 | |
| *** speedmann has joined #openstack-ansible | 07:18 | |
| *** gillesMo has joined #openstack-ansible | 07:30 | |
| *** blinkiz has quit IRC | 07:34 | |
| *** suggestable has joined #openstack-ansible | 07:36 | |
| *** shardy has joined #openstack-ansible | 07:37 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_zun master: tests: Ensure clouds.yaml is present, and use it https://review.openstack.org/586145 | 07:47 |
| *** shyambiradar has quit IRC | 07:47 | |
| *** jwitko_ has quit IRC | 07:48 | |
| *** shyambiradar has joined #openstack-ansible | 07:48 | |
| *** gkadam has joined #openstack-ansible | 07:49 | |
| *** rpittau has quit IRC | 07:57 | |
| *** rpittau has joined #openstack-ansible | 07:57 | |
| gillesMo | Hello ! Is it safe to "apt upgrade" all the hosts and containers ? I don't see any doc about system upgrades. To push to the limit, how ar we supposed to upgrade from Ubuntu 16.04 to Ubuntu 18.04 ? Add new nodes, remove old ? Inplace upgrade ? What difference between hosts and LXC containers ? I've also posted this question here : https://ask.opens | 07:58 |
| gillesMo | tack.org/en/question/115470/how-to-handle-system-upgrades-using-openstack-ansible/ | 07:58 |
| odyssey4me | gillesMo: I don't have the ether pad on-hand, perhaps others will when they come online later today | 08:04 |
| odyssey4me | I don't that logan- and mgariepy have done the trusty->xenial move, so they could help you with a major system upgrade. | 08:04 |
| odyssey4me | *I know | 08:04 |
| odyssey4me | OSA does not yet support Ubuntu 18.0.4 yet, so do not do that. | 08:05 |
| odyssey4me | But yes, as far as anything else goes - you can upgrade the system packages as you wish - just don't do Xenial -> Bionic. | 08:05 |
| gillesMo | odyssey4me: OK, sure I don't want to upgrade NOW to bionic, but I would like to know how I will handle that in the futur, when xenial will not be supported any more by openstack-ansible ! | 08:12 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_blazar master: Remove unnecessary RabbitMQ setup/vars https://review.openstack.org/586472 | 08:12 |
| odyssey4me | gillesMo: Well, we first need to get it working with bionic. Work has started, but we need more hands to get it done. Given the lack of momentum of it for the moment, it's likely that will only get done in Stein. | 08:13 |
| odyssey4me | And sure, hopefully this time we can publish a guide in the upgrade docs. | 08:14 |
| *** shyambiradar has quit IRC | 08:16 | |
| *** lukaslemes has joined #openstack-ansible | 08:44 | |
| *** shyamb has joined #openstack-ansible | 08:48 | |
| *** flaviosr has quit IRC | 09:01 | |
| *** flaviosr has joined #openstack-ansible | 09:02 | |
| odyssey4me | gillesMo: Of course you could contribute a playbook for that into the openstack-ansible-ops repo. That repo is for exactly that purpose. | 09:04 |
| odyssey4me | (for operators to contribute things that are useful for their use-cases) | 09:04 |
| gillesMo | odyssey4me: Of course. I first try to see if it doesn't already exist somewhere ! | 09:06 |
| *** poul has joined #openstack-ansible | 09:11 | |
| odyssey4me | and boom - we have a working functional test for zun now :) https://review.openstack.org/586145 cloudnull | 09:11 |
| evrardjp | odyssey4me: woot | 09:12 |
| evrardjp | odyssey4me: you will bring https://review.openstack.org/#/c/583399/ on top ? | 09:13 |
| odyssey4me | evrardjp: yeah, I need to revise it - along with the others | 09:14 |
| odyssey4me | will do those today | 09:14 |
| evrardjp | want us to merge those quickly so you can iterate over the already merged things? | 09:15 |
| evrardjp | instead of being case by case? | 09:15 |
| odyssey4me | evrardjp: which - the hybrid messaging stuff? yes, that'd be nice | 09:15 |
| odyssey4me | I can script in the extra changes for all the other places where that's already merged. | 09:16 |
| evrardjp | that's my point | 09:18 |
| evrardjp | if we merge it everywhere it will be easier | 09:18 |
| *** poul has quit IRC | 09:18 | |
| odyssey4me | much, much easier :) | 09:20 |
| evrardjp | yeah but some have -1 from you | 09:21 |
| evrardjp | -w | 09:21 |
| evrardjp | https://review.openstack.org/#/q/topic:move-mq-setup-into-roles+(status:open+OR+status:merged) | 09:21 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_blazar master: Remove unnecessary inventory configuration https://review.openstack.org/586489 | 09:25 |
| odyssey4me | yes, those need the hybrid messaging patches merged first, then I can revise them | 09:25 |
| odyssey4me | All the 'Update the ...' patches here: https://review.openstack.org/#/q/topic:bp/hybrid-messaging+status:open | 09:26 |
| *** hachi has joined #openstack-ansible | 09:26 | |
| odyssey4me | and the swift one needs https://review.openstack.org/#/c/584630/ | 09:27 |
| evrardjp | let's blindly merge and fix? | 09:29 |
| *** taseer2 is now known as Taseer | 09:34 | |
| *** andymccr- has joined #openstack-ansible | 09:47 | |
| *** asettle- has joined #openstack-ansible | 09:47 | |
| *** andymccr_ has quit IRC | 09:50 | |
| *** asettle_ has quit IRC | 09:51 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_barbican master: Fix usage of "|" for tests https://review.openstack.org/582232 | 09:53 |
| odyssey4me | evrardjp: yes, for all of those left I've already done the fixes - for everything that's merged already, I'll script a series of patches to apply the fixes | 09:54 |
| jrosser | the versions of rabbitmq/erlang in the bionic tests dont seem to agree with https://www.rabbitmq.com/which-erlang.html | 09:57 |
| jrosser | and it's rabbitmq which blows up across the board for bionic | 09:57 |
| evrardjp | jrosser: wow that's bad | 10:00 |
| openstackgerrit | Merged openstack/openstack-ansible-os_watcher master: Update to use oslo.messaging service for RPC and Notify https://review.openstack.org/585736 | 10:01 |
| suggestable | Morning all | 10:02 |
| suggestable | When I run setup-hosts.yml (stable/ocata) on CentOS 7, I get this: | 10:03 |
| suggestable | fatal: [os-controller1]: FAILED! => {"changed": false, "cmd": ["btrfs", "quota", "disable", "/var/lib/machines"], "delta": "0:00:00.004725", "end": "2018-07-27 09:54:02.106861", "failed": true, "msg": "non-zero return code", "rc": 1, "start": "2018-07-27 09:54:02.102136", "stderr": "ERROR: not a btrfs filesystem: /var/lib/machines", "stderr_lines": ["ERROR: not a btrfs filesystem: /var/lib/machines"], "stdout": "", "stdout_lines": []} | 10:03 |
| suggestable | Followed by this a few lines down: | 10:03 |
| suggestable | TASK [lxc_hosts : Place container rootfs] ****************************************************************************************************************************************************************************************************************************************************************************************************************************** | 10:03 |
| suggestable | Friday 27 July 2018 09:54:05 +0100 (0:00:00.552) 0:01:51.887 *********** | 10:03 |
| suggestable | fatal: [os-controller1]: FAILED! => {"changed": false, "failed": true, "msg": "dest '/var/lib/machines/centos-7-amd64' must be an existing dir"} | 10:03 |
| *** asettle has quit IRC | 10:04 | |
| *** andymccr has quit IRC | 10:04 | |
| suggestable | They're the only two lines that fail (running the playbook limited to os-controller1). | 10:04 |
| suggestable | I've tried doing rm-rf /var/lib/machines and rm /var/lib/machines.raw and rebooting the target host, and still get the same errors. | 10:04 |
| *** andymccr- is now known as andymccr | 10:05 | |
| suggestable | (And yes, this means I've decided to use LXC containers) | 10:05 |
| *** asettle- is now known as asettle | 10:05 | |
| suggestable | Any ideas? | 10:05 |
| suggestable | (Current deployment packages are 17.0.8) | 10:06 |
| odyssey4me | suggestable: 17 is pike, but you're saying you're doing ocata - which is it? | 10:09 |
| suggestable | sorry... stable/queens | 10:10 |
| jrosser | evrardjp: checked in here for the rabbit/erlang versions http://logs.openstack.org/50/582150/1/check/openstack-ansible-functional-ubuntu-bionic/73d1838/logs/ubuntu-apt-list-installed-container1.txt.txt.gz | 10:14 |
| openstackgerrit | Merged openstack/openstack-ansible-os_congress master: Update to use oslo.messaging service for RPC and Notify https://review.openstack.org/585825 | 10:15 |
| openstackgerrit | Merged openstack/openstack-ansible-os_zun master: tests: Ensure clouds.yaml is present, and use it https://review.openstack.org/586145 | 10:19 |
| rpittau | suggestable: you have the error message right there, the script is expecting /var/lib/machines to be btrfs type filesystem | 10:19 |
| suggestable | rpittau: OK. Why does it not create this for me, like it's supposed to? | 10:20 |
| suggestable | I'm running on CentOS 7.5 | 10:20 |
| *** blinkiz has joined #openstack-ansible | 10:21 | |
| evrardjp | jrosser: is it due to hipe? | 10:22 |
| evrardjp | What's the recommended version ? | 10:23 |
| rpittau | suggestable: my understanding is that that is part of the bootstrap procedure: https://github.com/openstack/openstack-ansible/blob/stable/queens/tests/roles/bootstrap-host/tasks/prepare_loopback_machines.yml | 10:23 |
| blinkiz | Hello. If I already have a centralized logging server (graylog gui) setup, where can I specify openstack-ansible to use this | 10:23 |
| blinkiz | oh, sorry... pressed the enter key before writing everything :-P | 10:23 |
| odyssey4me | oh yes, 17 is queens | 10:23 |
| suggestable | rpittau: This is not an AIO deployment. | 10:23 |
| evrardjp | blinkiz: I started to have a thing that deploys graylog | 10:24 |
| evrardjp | so don't use that, but you can use it for inspiration | 10:24 |
| blinkiz | Hello. If I already have a centralized logging server (graylog gui) setup, where can I specify openstack-ansible to use this IP and port? Instead of setting up a lxc container for this.. | 10:24 |
| evrardjp | and improve it :) | 10:24 |
| *** lukaslemes has quit IRC | 10:24 | |
| odyssey4me | suggestable: not sure, cloudnull is probably best to advise when he's in | 10:24 |
| evrardjp | blinkiz: https://github.com/openstack/openstack-ansible-ops/blob/master/graylog/graylog-forward-logs.yml | 10:24 |
| suggestable | odyssey4me: Thank you Jesse | 10:25 |
| evrardjp | blinkiz: this was how I installed graylog: https://github.com/openstack/openstack-ansible-ops/blob/master/graylog/graylog2-install.yml | 10:25 |
| blinkiz | evrardjp, oh, experimenting with gelf.. I see :) | 10:25 |
| jrosser | evrardjp: it looks like for rabbit 3.7.5 you need erlang maximum 20.3.x | 10:25 |
| jrosser | but i didnt get as far as seeing how any of those versions came to be there | 10:26 |
| evrardjp | blinkiz: this was my PoC. In a few hours I got enough log sent , so I stopped my experimentation. I had enough to prove it's super easy to send logs with graylog! | 10:26 |
| evrardjp | this is the readme: https://github.com/openstack/openstack-ansible-ops/blob/master/graylog/README.rst | 10:26 |
| evrardjp | jrosser: can we bump rabbit more? | 10:26 |
| jrosser | 3.7.7 would go with the erlang version that is there | 10:27 |
| jrosser | according to the compatibility table | 10:27 |
| blinkiz | Okay.. But let's forget graylog for a while so I understand everything. Where is the configuration for specifying where all containers should send it's logs? I already have centralized log handling. | 10:28 |
| openstackgerrit | Merged openstack/openstack-ansible-os_masakari master: Update to use oslo.messaging service for RPC and Notify https://review.openstack.org/585526 | 10:28 |
| openstackgerrit | Merged openstack/openstack-ansible-os_barbican master: Update to use oslo.messaging service for RPC and Notify https://review.openstack.org/574864 | 10:28 |
| evrardjp | jrosser: yeah I am on the website | 10:28 |
| jrosser | evrardjp: actually it says it plainly at the top "RabbitMQ versions prior to 3.7.7 do not support Erlang/OTP 21 or newer" | 10:28 |
| evrardjp | I'd go for 3.7.7 | 10:28 |
| jrosser | yes that | 10:28 |
| evrardjp | jrosser: https://groups.google.com/forum/#!msg/rabbitmq-users/KbOldePfgYw/cjYzldEJAQAJ | 10:29 |
| evrardjp | fun | 10:29 |
| blinkiz | Read more a but your graylog gelf integration. Sounds promising. Probably gonna test that out :) | 10:29 |
| evrardjp | blinkiz: I think there are few issues around that | 10:30 |
| evrardjp | blinkiz: like some services don't log binarily | 10:30 |
| evrardjp | that's something to improve | 10:30 |
| blinkiz | mmm | 10:30 |
| evrardjp | but most of them do IIRC | 10:30 |
| *** speedmann has quit IRC | 10:31 | |
| evrardjp | it's been long ago I don't remember | 10:31 |
| evrardjp | IIRC it was horizon logs that were not so good | 10:31 |
| *** shyamb has quit IRC | 10:32 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_tacker master: Update to use oslo.messaging service for RPC and Notify https://review.openstack.org/585709 | 10:32 |
| *** shyamb has joined #openstack-ansible | 10:32 | |
| *** shyamb has quit IRC | 10:33 | |
| *** speedmann has joined #openstack-ansible | 10:34 | |
| jrosser | evrardjp: what do you make of http://www.rabbitmq.com/debian/ vs https://packagecloud.io/rabbitmq/rabbitmq-server ? if 3.7.7 is needed it looks like it's in the 2nd of those | 10:37 |
| *** savvas has quit IRC | 10:53 | |
| *** savvas has joined #openstack-ansible | 10:53 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tacker master: Move MQ vhost/user creation into role https://review.openstack.org/583394 | 11:03 |
| *** dave-mccowan has joined #openstack-ansible | 11:05 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tacker master: Move MQ vhost/user creation into role https://review.openstack.org/583394 | 11:06 |
| *** vnogin has joined #openstack-ansible | 11:10 | |
| *** speedmann has quit IRC | 11:11 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_watcher master: Move MQ vhost/user creation into role https://review.openstack.org/583397 | 11:11 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_watcher master: Move MQ vhost/user creation into role https://review.openstack.org/583397 | 11:13 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tacker master: Move MQ vhost/user creation into role https://review.openstack.org/583394 | 11:19 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tacker master: Move MQ vhost/user creation into role https://review.openstack.org/583394 | 11:21 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tacker master: Move MQ vhost/user creation into role https://review.openstack.org/583394 | 11:21 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_watcher master: Move MQ vhost/user creation into role https://review.openstack.org/583397 | 11:25 |
| *** cshen has quit IRC | 11:27 | |
| *** vnogin has quit IRC | 11:29 | |
| *** cshen has joined #openstack-ansible | 11:30 | |
| *** jamesde__ has joined #openstack-ansible | 11:31 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_congress master: Move MQ vhost/user creation into role https://review.openstack.org/583369 | 11:32 |
| *** jamesden_ has quit IRC | 11:32 | |
| *** jamesde__ has quit IRC | 11:34 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Move MQ vhost/user creation into role https://review.openstack.org/583380 | 11:37 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_zun master: Move MQ vhost/user creation into role https://review.openstack.org/583399 | 11:42 |
| *** hachi has quit IRC | 11:43 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_masakari master: Add the missing "#" https://review.openstack.org/583991 | 11:43 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_congress master: Move MQ vhost/user creation into role https://review.openstack.org/583369 | 11:44 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_barbican master: Move MQ vhost/user creation into role https://review.openstack.org/583363 | 11:51 |
| openstackgerrit | Merged openstack/openstack-ansible-os_masakari master: Remove the unnecessary = https://review.openstack.org/583999 | 11:56 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tacker master: Remove the unnecessary space https://review.openstack.org/582818 | 12:01 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tacker master: Remove the unnecessary = https://review.openstack.org/583996 | 12:01 |
| openstackgerrit | Merged openstack/openstack-ansible master: Add jmespath to requirements.txt https://review.openstack.org/586331 | 12:02 |
| *** savvas has quit IRC | 12:04 | |
| *** vnogin has joined #openstack-ansible | 12:05 | |
| evrardjp | jrosser: well packagecloud is recommended on rabbitmq website | 12:07 |
| *** vnogin has quit IRC | 12:08 | |
| *** savvas has joined #openstack-ansible | 12:09 | |
| *** savvas has quit IRC | 12:11 | |
| *** savvas_ has joined #openstack-ansible | 12:11 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests master: Remove all MQ vhost/user creation tasks/vars https://review.openstack.org/586524 | 12:19 |
| odyssey4me | evrardjp: ^ nice cruft cleanup, -404 lines | 12:20 |
| *** armaan has joined #openstack-ansible | 12:22 | |
| evrardjp | odyssey4me: I'd like to wait for m3 to pass first :p | 12:22 |
| evrardjp | but yeah. | 12:22 |
| odyssey4me | that's tests repo, so won't affect the itegrated build | 12:22 |
| evrardjp | there are many things we will be able to clean up | 12:23 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_aodh master: Remove unnecessary MQ vhost/user creation tasks/vars https://review.openstack.org/586529 | 12:30 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_ceilometer master: Remove unnecessary MQ vhost/user creation tasks/vars https://review.openstack.org/586530 | 12:33 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_designate master: Remove unnecessary MQ vhost/user creation tasks/vars https://review.openstack.org/586532 | 12:37 |
| *** savvas_ has quit IRC | 12:40 | |
| *** armaan has quit IRC | 12:41 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Add swift_oslomsg_notify_password https://review.openstack.org/584630 | 12:41 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_magnum master: Remove unnecessary MQ vhost/user creation tasks/vars https://review.openstack.org/586534 | 12:41 |
| noonedeadpunk | hi everyone | 12:42 |
| noonedeadpunk | odyssey4me please, let me know if you'll have time for masakari) | 12:42 |
| *** armaan has joined #openstack-ansible | 12:45 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_trove master: Remove unnecessary MQ vhost/user creation tasks/vars https://review.openstack.org/586535 | 12:45 |
| *** savvas has joined #openstack-ansible | 12:45 | |
| openstackgerrit | Dmitriy Rabotjagov (noonedeadpunk) proposed openstack/openstack-ansible-os_horizon master: Implements custom theme distribution https://review.openstack.org/586318 | 12:45 |
| odyssey4me | noonedeadpunk: ok, let me take a look now | 12:46 |
| noonedeadpunk | odyssey4me it's about https://review.openstack.org/#/c/584629/ | 12:47 |
| blinkiz | In my lab environment I have done something so all containers is complaining about not being able to connect to mysql. Where do I start to troubleshoot this? | 12:49 |
| *** armaan has quit IRC | 12:49 | |
| noonedeadpunk | it seems, that I need to rebase to the new master and resolve conflicts... But lets cross the bridge, when I come to it. | 12:50 |
| *** savvas has quit IRC | 12:50 | |
| *** armaan has joined #openstack-ansible | 12:50 | |
| odyssey4me | evrardjp: given that all our role upgrade tests are failing, do you think we should just remove them? | 12:50 |
| odyssey4me | they're currently just cruft | 12:50 |
| *** ansmith has joined #openstack-ansible | 12:52 | |
| jrosser | evrardjp: i dig some more - there is no bionic version of erlang which meets the pin set in the rabbitmq server role | 12:53 |
| jrosser | so it looks like the pin gets set in apt preferences but doesnt appear to do anything | 12:53 |
| odyssey4me | ansmith: good morning, looks like monasca and cloudkitty are still outstanding for the hybrid messaging changes - any chance that you could get those done today? | 12:53 |
| *** armaan has quit IRC | 12:54 | |
| *** savvas has joined #openstack-ansible | 12:54 | |
| ansmith | odyssey4me: yes, I had cloudkitty done and then got distracted, will hit monasca this morning | 12:54 |
| odyssey4me | ansmith: awesome, thanks | 12:55 |
| evrardjp | odyssey4me: that's harsh for my heart. Nobody is working on those, nor on the integrated upgrades. | 12:56 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_searchlight master: Remove unnecessary RabbitMQ setup/vars https://review.openstack.org/586539 | 12:57 |
| odyssey4me | evrardjp: the role upgrades can't work, ever again - we need to use the integrated build for upgrades | 12:58 |
| *** lirt has joined #openstack-ansible | 12:58 | |
| odyssey4me | so my vote is to remove them from the roles for now - the re-instate them once we're using the integrated role for tests | 12:58 |
| lirt | Hey guys, can someone point me how to hotfix https://bugs.launchpad.net/openstack-ansible/+bug/1760574 ? Until fix is released | 12:59 |
| openstack | Launchpad bug 1760574 in openstack-ansible "pip2 fails to install pkgs requiring python3 (dnspython3, mypy, etc)" [Critical,Fix released] | 12:59 |
| *** hachi has joined #openstack-ansible | 12:59 | |
| *** savvas has quit IRC | 12:59 | |
| ansmith | odyssey4me, evrardjp : Could use some insight when you have a chance. When messaging backend is other than rabbitmq, need to setup extra packages such as in https://review.openstack.org/#/c/584450/ | 12:59 |
| *** rmart04 has joined #openstack-ansible | 12:59 | |
| odyssey4me | lirt: that fix landed ages ago | 12:59 |
| odyssey4me | lirt: just use the most recent stable branch or a more recent release | 13:00 |
| evrardjp | ansmith: what's the issue? | 13:00 |
| ansmith | odyssey4me, evrardjp: initial issue was version of oslo.messaging that supported the extras, that is now fixed and I see packages in the repo but during service install, the file is not found | 13:00 |
| odyssey4me | ansmith: hmm, ok - does the task that installs nova_optional_oslomsg_amqp1_pip_packages ever run? | 13:01 |
| ansmith | evrardjp: you were helping me with this a week back when it was the oslo.messaging issue, example where qdrouterd is backend https://review.openstack.org/#/c/570261/ | 13:01 |
| lirt | @odyssey4me Should I use this pip https://bootstrap.pypa.io/3.3/get-pip.py? | 13:01 |
| odyssey4me | lirt: if you're using a recent release then that will be use automatically | 13:02 |
| noonedeadpunk | odyssey4me so my problem is, that for some reason pip tries to uninstall python-libvirt, and of course fails (as it was linked from outside of venv). | 13:02 |
| noonedeadpunk | odyssey4me seems like issue in some extra_args (as changing state from latest t opresent doesn't help). And masakari-monitor doesn't require python-libvirt of older version | 13:03 |
| odyssey4me | lirt: ah, I see we haven't release a version that includes it yet - please use stable/queens | 13:03 |
| lirt | So go for master? I am on 17.0.7 | 13:05 |
| lirt | Sorry stable/queens. OK | 13:05 |
| ansmith | odyssey4me, evrardjp: yes the task runs, for example, glance optional install here http://logs.openstack.org/61/570261/14/check/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/e12f64e/job-output.txt.gz | 13:05 |
| evrardjp | ansmith: fyi it's often easier to grok with the ara web interface: http://logs.openstack.org/61/570261/14/check/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/e12f64e/logs/ara-report/ | 13:06 |
| evrardjp | ansmith: we still have to figure out why 6.5 is pulled there | 13:07 |
| evrardjp | no ? | 13:07 |
| evrardjp | http://logs.openstack.org/61/570261/14/check/openstack-ansible-deploy-aio_lxc-ubuntu-xenial/e12f64e/logs/openstack/aio1_repo_container-21dc0f6f/repo/wheel_build.log | 13:07 |
| *** mgariepy has quit IRC | 13:08 | |
| evrardjp | odyssey4me: didn't we say we should log the text files of the repo server? | 13:08 |
| ansmith | evrardjp, yes, that is still to be figured out, but at least 6.5 supports the extra packages, last week, we were using 6.0 | 13:09 |
| evrardjp | oh ok | 13:09 |
| odyssey4me | evrardjp: yeah, let me work up a patch for that - it keeps coming up | 13:10 |
| evrardjp | odyssey4me: ok | 13:10 |
| evrardjp | ansmith: odyssey4me in the meantime I will run a node with your patch, see what's going on | 13:10 |
| *** mgariepy has joined #openstack-ansible | 13:10 | |
| ansmith | evrardjp: pyngus was the dependency we were missing and it is now in the log, is there a step I am missing to get it into the service venv iiuc? | 13:11 |
| odyssey4me | ansmith: no, seems like it should do the right things - but isn't | 13:12 |
| evrardjp | mmm is it really pyngus missing? | 13:12 |
| evrardjp | it looks okay | 13:12 |
| ansmith | this is the failure -> fatal: [aio1_glance_container-4f27f62f]: FAILED! => {"attempts": 5, "changed": false, "cmd": "/openstack/venvs/glance-18.0.0.0b3/bin/pip2 install -U 'oslo.messaging[amqp1]'", "msg": "[Errno 2] No such file or directory", "rc": 2} | 13:13 |
| odyssey4me | no such file or directory - that means it can't find pip2 | 13:15 |
| odyssey4me | or perhaps the glance venv | 13:15 |
| odyssey4me | is the ordering right? | 13:15 |
| *** tux_ has joined #openstack-ansible | 13:15 | |
| odyssey4me | yep, looks like it is | 13:16 |
| *** savvas has joined #openstack-ansible | 13:16 | |
| odyssey4me | hang on a sec, looks like some tasks are missing | 13:16 |
| odyssey4me | ah yes, ordering is wrong here https://review.openstack.org/#/c/584450/1/tasks/nova_install_source.yml | 13:17 |
| odyssey4me | that new task must be just before 'Record the venv tag deployed', otherwise the venv is not ready yet | 13:17 |
| odyssey4me | the task 'Update virtualenv path' must run before anything tries to use the venv | 13:18 |
| odyssey4me | ansmith evrardjp ^ | 13:18 |
| *** tux__ has joined #openstack-ansible | 13:19 | |
| evrardjp | odyssey4me: that's exactly what I thought: this no such file or directory is very shebang issue like | 13:19 |
| evrardjp | you've confirmed it before I could confirm it with my eyes! | 13:19 |
| evrardjp | odyssey4me: do you update? | 13:20 |
| evrardjp | I can update the task | 13:20 |
| evrardjp | while you fix the gathering of txt files :D | 13:20 |
| odyssey4me | evrardjp: we can leave it to ansmith to update - there are a lot of those patches ;) | 13:20 |
| evrardjp | haha ok | 13:21 |
| *** savvas has quit IRC | 13:21 | |
| odyssey4me | some of which have merged :p | 13:21 |
| evrardjp | ansmith: got it? | 13:21 |
| *** savvas has joined #openstack-ansible | 13:21 | |
| ansmith | odyssey4me, evrardjp: perfect thanks! not sure i would have figure that one. Yes I will update and retry | 13:21 |
| evrardjp | ansmith: I will explain so you understand the issue: We build in the repo build with a specific path, that gets updated with a task "Update virtualenv path" to be prod appropriate (not /tmp/whatever) | 13:22 |
| *** tux_ has quit IRC | 13:22 | |
| evrardjp | when "virtualenv" is passed to the pip task, the python from the virtualenv is used instead of ansible_python_interpreter | 13:23 |
| evrardjp | so because it's not updated yet, it would fail with no such file or directory | 13:23 |
| odyssey4me | evrardjp: actually, I need to get on looking into the masakari role - could you figure out the gathering of the repo build os-releases/*.txt files? | 13:23 |
| evrardjp | ok | 13:23 |
| odyssey4me | not urgent, obviously - can always do it later, but if you have a gap then go for it | 13:24 |
| evrardjp | I prefer making a gap for this | 13:24 |
| evrardjp | it's so painful | 13:24 |
| ansmith | evrardjp: Thanks for the explanation, very helpful. I will move the optional install to after the update venv path. A fun way to spend Friday :) | 13:25 |
| *** vnogin has joined #openstack-ansible | 13:25 | |
| odyssey4me | noonedeadpunk: ok, let me get this straight - the general idea is to run the monitors on compute hosts - right? do they run on any others? | 13:28 |
| noonedeadpunk | odyssey4me: they are supposed to run on compute hosts - it doesn't has much sense to run them elsewhere | 13:28 |
| odyssey4me | noonedeadpunk: ok, then I suggest we change this test to actually deploy nova so that you have a compute host there already | 13:29 |
| noonedeadpunk | as they should monitor whether all compute node are up and trigger nova evacuate if one node is down | 13:29 |
| odyssey4me | noonedeadpunk: if you don't mind, I can fire up a test node and take a crack at it for you - then update the patch | 13:31 |
| noonedeadpunk | odyssey4me: so nova installation should be held on bare metal? | 13:31 |
| odyssey4me | once it's working, I can explain every change | 13:31 |
| odyssey4me | noonedeadpunk: yes, definitely | 13:31 |
| noonedeadpunk | odyssey4me: of course I don't mind. However, problem appears, when masakari-monitors pip package gets installed into virtualenv by "Install pip packages" task from masakari_install. | 13:32 |
| noonedeadpunk | odyssey4me And manual installation of package runs smoothly, and pip is fully satisfied by linked python-libvirt | 13:33 |
| odyssey4me | yeah, that's kinda why I'd like to take a crack at it all at once... it's a solvable problem, but a branch new one which we haven't yet seen | 13:34 |
| odyssey4me | well, hang on - masakari-monitors as a package doesn't need to be in the normal venv, does it? | 13:34 |
| odyssey4me | the service venvs I mean | 13:35 |
| noonedeadpunk | odyssey4me: yep, no problem at all. Also my patch is currently conflicting with master (after merging another patches). | 13:35 |
| odyssey4me | yeah, I can fix that real quick | 13:35 |
| odyssey4me | ok, let me pull it down and have a go - happy friday :) | 13:35 |
| evrardjp | odyssey4me: we seem to log many things in the roles directly in the appropriate /var/log/ (like the wheel building). I am enclined to do log the requirements and other .txt there. opinion? | 13:36 |
| noonedeadpunk | odyssey4me: not fully understand about normal venv. Are you going to install it outside of venv? | 13:36 |
| *** pkaralis has quit IRC | 13:38 | |
| noonedeadpunk | odyssey4me: don't mind. do everything you would like to | 13:38 |
| *** blinkiz has quit IRC | 13:41 | |
| *** tux__ has quit IRC | 13:48 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-repo_build master: Log repo files https://review.openstack.org/586555 | 13:48 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server master: Bump rabbitmq version to 3.7.7 for bionic https://review.openstack.org/586558 | 13:50 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Log repo files https://review.openstack.org/586559 | 13:50 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server master: Bump rabbitmq version to 3.7.7 for bionic https://review.openstack.org/586558 | 13:53 |
| *** ianychoi has joined #openstack-ansible | 13:56 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Add missing masakari_venv_download_url default https://review.openstack.org/586565 | 13:58 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Use masakari-api group for setup tasks https://review.openstack.org/586566 | 13:58 |
| *** ianychoi_ has quit IRC | 14:00 | |
| odyssey4me | evrardjp: a far simpler mechanism would be to copy the file from /openstack/<repo container name>/repo/os-releases/*/*.txt | 14:01 |
| odyssey4me | it's already o the host - no need to do funky stuff in the repo_build role | 14:02 |
| odyssey4me | the only exception would be the metal build, where you'd have to grab it from /var/www/repo/... | 14:02 |
| *** cshen has quit IRC | 14:03 | |
| evrardjp | odyssey4me: well I thought this was where the things are logged already currently | 14:05 |
| evrardjp | odyssey4me: tbh I wanted to do this with a proper play | 14:06 |
| evrardjp | that we run only in gates. | 14:06 |
| odyssey4me | nvm, lemme push a patch up quick | 14:07 |
| *** FrankZhang has joined #openstack-ansible | 14:08 | |
| evrardjp | well | 14:09 |
| evrardjp | if it's to completely discard my perfectly working solution, I'd rather have not done it in the first place! | 14:10 |
| evrardjp | you didn't answer too : p | 14:10 |
| evrardjp | #whereisthecomplaintsofficewhenweneedit? | 14:10 |
| evrardjp | :p | 14:11 |
| odyssey4me | well, what's been done in your patches is over complicated and affect production | 14:11 |
| odyssey4me | we already only collect logs in gates | 14:11 |
| evrardjp | in that ugly bash function in the scripts library? | 14:12 |
| *** hachi has quit IRC | 14:14 | |
| odyssey4me | yes, and you can feel free to go ahead and rewrite that | 14:15 |
| evrardjp | odyssey4me: I'd rather we re-write this independently of the is_metal not is_metal by doing a proper play | 14:15 |
| odyssey4me | but just remember that it collects from even before there is ansible present | 14:15 |
| odyssey4me | go ahead for that - I'll do a stop-gap | 14:16 |
| evrardjp | odyssey4me: we have zuul's ansible. | 14:16 |
| evrardjp | if need be | 14:16 |
| odyssey4me | yeah, fair enough - assuming that we only ever want to collect logs in zuul | 14:16 |
| evrardjp | but what do you mean by it collects from even before there is ansible present? | 14:17 |
| *** goldenfri has joined #openstack-ansible | 14:17 | |
| odyssey4me | if our build fails before ansible is there, we still need logs | 14:17 |
| *** vnogin has quit IRC | 14:17 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Collect the generated repo_build files https://review.openstack.org/586572 | 14:17 |
| *** shardy has quit IRC | 14:18 | |
| odyssey4me | if logan- is good with collecting logs via an ansible play (given he has external CI) then that's a good idea | 14:18 |
| cloudnull | mornings | 14:19 |
| *** tux_ has joined #openstack-ansible | 14:19 | |
| evrardjp | too much energy spent on this already | 14:19 |
| evrardjp | abandonning my patches. | 14:19 |
| eumel8 | o/ | 14:21 |
| cloudnull | odyssey4me nice work on the zun functional testing, that's great! | 14:22 |
| eumel8 | just wondering, in openstack-ansible-haproxy_server is only an option for self-signed certificates. Is there a reason for that? An option for letsencrypt certs would be nice. | 14:24 |
| mgariepy | eumel8, you can configure your cert with it, not only self signed ones | 14:27 |
| mgariepy | eumel8, https://github.com/openstack/openstack-ansible-haproxy_server/blob/master/tasks/haproxy_ssl_user_provided.yml#L18 | 14:28 |
| odyssey4me | eumel8: it would be, unfortunately no-one has stepped up to submit a patch to make it happen | 14:31 |
| eumel8 | mgariepy: I know. I used before ansible-playbook haproxy-install.yml -e @/home/ubuntu/cert.yml with my own certs but now I have a fresh installation and no valid certs anymore. So my idea was to use letsencrypt for our own community server | 14:31 |
| mgariepy | eumel8, you can generate the cert somewhere and the push it via osa ? | 14:32 |
| eumel8 | odyssey4me: thanks for giving the chance ;) | 14:32 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Use masakari_etc_dir consistently https://review.openstack.org/586579 | 14:32 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Use masakari_log_dir consistently https://review.openstack.org/586580 | 14:32 |
| eumel8 | mgariepy: yeah, this would be the best option for the moment | 14:33 |
| mgariepy | eumel8, https://docs.ansible.com/ansible/2.5/modules/letsencrypt_module.html | 14:33 |
| mgariepy | it would be really nice to have the cert generated / propagated automatically tho, if you have ideas on how it could work i am interested, i'm just not quite sure how it could be done easily right now. | 14:34 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Use masakari_etc_dir consistently https://review.openstack.org/586579 | 14:35 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Use masakari_log_dir consistently https://review.openstack.org/586580 | 14:35 |
| eumel8 | mgariepy: ah, thx for the link! | 14:35 |
| mgariepy | eumel8, np. | 14:36 |
| mgariepy | eumel8, if you write a playbook and want to share it after , please let me know | 14:37 |
| FrankZhang | odyssey4me hmm seems this also ran into dragonflow issue as well: https://review.openstack.org/#/c/586269/ | 14:37 |
| eumel8 | mgariepy: ok | 14:37 |
| odyssey4me | FrankZhang: yeah, I need to make a build and check it out - I should have kicked that off ages ago - sorry, have been busy with other things | 14:38 |
| odyssey4me | I'll kick one off in the background now | 14:38 |
| FrankZhang | nice thanks | 14:38 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Use masakari_service_* vars consistently https://review.openstack.org/586585 | 14:39 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_cloudkitty master: Update to use oslo.messaging service RPC and Notify https://review.openstack.org/586586 | 14:39 |
| *** shardy has joined #openstack-ansible | 14:40 | |
| cloudnull | anyone around want to give https://review.openstack.org/#/c/585524/ a push forward ? | 14:41 |
| mgariepy | cloudnull, on it | 14:41 |
| cloudnull | if approved i'll start replicating that change to the other os_* roles | 14:41 |
| cloudnull | thanks mgariepy | 14:41 |
| *** tux__ has joined #openstack-ansible | 14:44 | |
| *** tux_ has quit IRC | 14:44 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Convert logstash groks to a multi-pipeline setup https://review.openstack.org/586419 | 14:44 |
| tux__ | OSA using galera cluster what if one of node falling behind ? how does LB knows about it if its just doing TCP load-balancing? | 14:45 |
| cloudnull | tux__ it uses a sync check | 14:45 |
| cloudnull | on port 9200 | 14:45 |
| mgariepy | cloudnull, are the logs shipped to rsyslog server also with systemd-journal ? | 14:45 |
| cloudnull | yes using journal-remote | 14:46 |
| tux__ | cloudnull: So if they are not in sync what will happened, just documenting stuff of my cloud so asking stupid question | 14:46 |
| cloudnull | mgariepy https://github.com/openstack/openstack-ansible/blob/master/playbooks/infra-journal-remote.yml | 14:47 |
| mgariepy | ok cloudnull thanks | 14:47 |
| cloudnull | tux__ if the sync check fails the node will be demoted by hap | 14:47 |
| cloudnull | and one of the other nodes in the pool, that is in sync, will pick up the load | 14:48 |
| tux__ | so HAproxy doing sync check right on port 9200 and it will remove bad node? | 14:48 |
| suggestable | Hey guys. I managed to get past the previous roadblock, but now get this when running setup_infrastructure.yml | 14:48 |
| suggestable | fatal: [os-controller3_galera_container-5b27080e]: FAILED! => {"changed": false, "failed": true, "msg": "The galera_cluster_name variable does not match what is set in mysql. Check your galera_cluster_name setting in your user_*.yml files in \"/etc/openstack_deploy\" and compare to the current value in \"/etc/mysql/conf.d/cluster.cnf\" on the host, and the \"wsrep_cluster_name\" value set in your running galera cluster. To ignore the clus | 14:48 |
| suggestable | ter state set '-e galera_ignore_cluster_state=true'.\n"} | 14:48 |
| suggestable | I'm running stable/queens (17.0.8) | 14:49 |
| cloudnull | tux__ Yes. | 14:49 |
| tux__ | cloudnull: thanks i will check config to see what it doing it | 14:49 |
| jproulx | walking through 1st OSA test-deploy (mitaka-eol unfortunately) & have what's probably an LXC noob question: | 14:49 |
| jproulx | lxcbr0 exists with ip 10.0.3.1/24, setup-hosts.yml creates containers and puts their <id>_eth0 interfaces on lxcbr0 and <id>_eth1 on br-mgmt, so far so good. | 14:49 |
| jproulx | inside containers eth1 has address but eth0 does not (I'm assuming outbound net for package install is meant to be through eth0?) do I need to get ip on eth0 (how?) or arrange a router with NAT on br-mgmt? | 14:49 |
| cloudnull | suggestable have you added `-e galera_ignore_cluster_state=true'` to the command line ? | 14:49 |
| cloudnull | jproulx likely an issue with dnsmasq | 14:50 |
| jproulx | lol everything is an issue with dnsmasq | 14:50 |
| suggestable | cloudnull: Is that required during initial deployment? | 14:50 |
| cloudnull | eth0 should get dhcp from the host | 14:50 |
| jproulx | that was my expectation, hmmm | 14:50 |
| cloudnull | eth1 should be statically defined | 14:50 |
| cloudnull | suggestable no but if the deployment is in an odd state you may need to | 14:51 |
| cloudnull | the role checks the state of the environment before running anything | 14:51 |
| cloudnull | which is done to make sure we;re not causing breakage on an unhealthy cluser. | 14:52 |
| cloudnull | **cluster. | 14:52 |
| * cloudnull needs coffee | 14:52 | |
| jproulx | cloudnull: should have know as usual dnsmasq is conflicting with our standard local caching proxy, I've been through this one before... | 14:52 |
| * jproulx also need coffee apparently | 14:52 | |
| suggestable | cloudnull: Trying with that command line override now. | 14:52 |
| odyssey4me | cores, I have a series of patches up for masakari which it would be handy to get merged pronto to make the follow up work simpler: https://review.openstack.org/#/q/project:openstack/openstack-ansible-os_masakari+owner:%22Jesse+Pretorius+(odyssey4me)+%253Cjesse.pretorius%2540rackspace.co.uk%253E%22+status:open | 14:58 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_cinder master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/582632 | 15:01 |
| suggestable | cloudnull: Now I get a different error (different cluster member): | 15:07 |
| suggestable | failed: [os-controller2_galera_container-d6b51cea -> 10.10.18.237] (item=os-controller2_galera_container-d6b51cea) => {"attempts": 6, "changed": false, "failed": true, "item": "os-controller2_galera_container-d6b51cea", "msg": "Unable to start service mysql: Job for mariadb.service failed because the control process exited with error code. See \"systemctl status mariadb.service\" and \"journalctl -xe\" for details.\n"} | 15:07 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_nova master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584450 | 15:07 |
| odyssey4me | noonedeadpunk: to confirm, the masakari monitor hosts only need masakari-monitors and systemd-python in the venv, right? | 15:07 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_keystone master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584479 | 15:10 |
| noonedeadpunk | odyssey4me: yep. and python-libvirt, but it will be linked there | 15:11 |
| *** lirt has quit IRC | 15:12 | |
| odyssey4me | noonedeadpunk: also, is there a special reason why you put the monitorc configs in a seperate directory? | 15:12 |
| odyssey4me | I mean, they seem to share much the same .conf options - perhaps one is just a subset of the other? | 15:13 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_glance master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584496 | 15:13 |
| noonedeadpunk | odyssey4me: by default, masakari-monitors looks for configs in this directory. | 15:13 |
| noonedeadpunk | odyssey4me: as they are not really stable, I decided just to stick to defaults | 15:14 |
| odyssey4me | noonedeadpunk: ah, I see https://github.com/openstack/masakari-monitors/tree/master/etc/masakarimonitors | 15:14 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_heat master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584513 | 15:25 |
| *** vollman has quit IRC | 15:26 | |
| *** asettle has quit IRC | 15:34 | |
| *** andymccr has quit IRC | 15:34 | |
| *** andymccr has joined #openstack-ansible | 15:35 | |
| *** asettle has joined #openstack-ansible | 15:35 | |
| *** asettle is now known as Guest26652 | 15:35 | |
| -openstackstatus- NOTICE: A zuul config error slipped through and caused a pile of job failures with retry_limit - a fix is being applied and should be back up in a few minutes | 15:36 | |
| *** ChanServ changes topic to "A zuul config error slipped through and caused a pile of job failures with retry_limit - a fix is being applied and should be back up in a few minutes" | 15:36 | |
| *** gkadam has quit IRC | 15:38 | |
| *** cshen has joined #openstack-ansible | 15:40 | |
| noonedeadpunk | odyssey4me btw it also missing haproxy configuration, as I thought to add it in the next commit (like change auth in config to vars) | 15:42 |
| *** lvdombrkr89 has quit IRC | 15:45 | |
| *** gyee has joined #openstack-ansible | 15:52 | |
| *** gkadam has joined #openstack-ansible | 15:53 | |
| *** cshen has quit IRC | 15:54 | |
| *** gkadam has quit IRC | 15:54 | |
| *** gkadam has joined #openstack-ansible | 15:55 | |
| *** rpittau has quit IRC | 15:58 | |
| *** shardy has quit IRC | 16:01 | |
| *** openstackgerrit has quit IRC | 16:04 | |
| *** suggestable has quit IRC | 16:06 | |
| *** openstackgerrit has joined #openstack-ansible | 16:09 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible master: [TEST] bionic https://review.openstack.org/586607 | 16:09 |
| *** cshen has joined #openstack-ansible | 16:10 | |
| *** lbragstad_ is now known as lbragstad | 16:11 | |
| *** Darcidride has joined #openstack-ansible | 16:11 | |
| *** cshen has quit IRC | 16:14 | |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_aodh master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584899 | 16:15 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Basic implementation of masakari-monitors https://review.openstack.org/584629 | 16:17 |
| *** SimAloo has joined #openstack-ansible | 16:18 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Basic implementation of masakari-monitors https://review.openstack.org/584629 | 16:21 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_ceilometer master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584904 | 16:22 |
| openstackgerrit | David Wilde proposed openstack/openstack-ansible-ops master: Deploy ELK in MNAIO https://review.openstack.org/586612 | 16:22 |
| *** yolanda has quit IRC | 16:22 | |
| noonedeadpunk | odyssey4me: I was thinking about retrieving corosync interface dynamically, like this http://paste.openstack.org/show/726597/ but I never get into IF statement for some reason (nevertheless, all variables has expected values) | 16:23 |
| odyssey4me | noonedeadpunk: I haven't tried that patch update yet - will have a go now. | 16:24 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Basic implementation of masakari-monitors https://review.openstack.org/584629 | 16:25 |
| *** yolanda has joined #openstack-ansible | 16:29 | |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_designate master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584921 | 16:29 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Basic implementation of masakari-monitors https://review.openstack.org/584629 | 16:31 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Basic implementation of masakari-monitors https://review.openstack.org/584629 | 16:32 |
| jrosser | whats happened here to give verified -1 ? https://review.openstack.org/#/c/586558/ | 16:33 |
| jrosser | oh, is it the rest of the chesks are in progress? | 16:34 |
| noonedeadpunk | jrosser I think its related to openstackstatus and zuul problems | 16:34 |
| noonedeadpunk | jrosser recheck is required... | 16:35 |
| odyssey4me | yeah, CI system broke - not sure if it's fixed yet | 16:35 |
| odyssey4me | usually when it's fixed they'll clear the notice | 16:35 |
| * odyssey4me points at channel topic | 16:35 | |
| * jrosser notices it :) | 16:36 | |
| jrosser | been dipping in/out today a bit | 16:36 |
| tux__ | cloudnull: I am looking at F5 LB to monitor Galera cluster sync status | 16:37 |
| tux__ | create ltm monitor mysql /RPC/RPC_MON_GALERA { count 1 database information_schema debug no defaults-from mysql destination *:* interval 3 recv big5_chinese_ci recv-column 2 recv-row 0 send "select * from CHARACTER_SETS;" time-until-up 0 timeout 10 username monitoring } | 16:37 |
| tux__ | This is the check here for F5 | 16:37 |
| tux__ | I am not seeing anything interesting which tell LB MySQL is out of sync and remove from LB | 16:38 |
| noonedeadpunk | what do you think about this change https://review.openstack.org/#/c/586318/ ? | 16:38 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_ironic master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584952 | 16:40 |
| tux__ | cloudnull: interesting i can see them in HAProxy "server ostack-controller-01_galera_container-966ec0a6 192.168.100.88:3306 check port 9200 inter 12000 rise 2 fall 2 backup" | 16:41 |
| tux__ | Why they are not in F5 (i use RPC script to convert them for F5) | 16:41 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Basic implementation of masakari-monitors https://review.openstack.org/584629 | 16:41 |
| jamesdenton | tux__ that healthcheck on the F5 performs a select against a table in mysql. If that fails, it pulls the member | 16:42 |
| tux__ | jamesdenton: good to see you again, I got your point but i am trying to find out how F5 will understand if one of Galera node falling behind ? | 16:43 |
| jamesdenton | it won't. Though I'm not sure the haproxy check reveals that, either | 16:44 |
| tux__ | cloudnull: saying it use 9200 port to check status of sync and remove node if falling behind | 16:44 |
| tux__ | "server ostack-controller-03_galera_container-b1a3dceb 192.168.100.246:3306 check port 9200 inter 12000 rise 1 fall 1" | 16:45 |
| tux__ | This is the line i found in HAProxy | 16:45 |
| *** rmart04 has quit IRC | 16:46 | |
| tux__ | If HAProxy also not checking that, in that case i need to setup custom monitor to find out if any node falling behind or out of sync.. | 16:47 |
| jamesdenton | i'm not sure what 9200 reveals, unless it's some built-in thing to not respond on 9200 if its out of sync | 16:47 |
| tux__ | You guys never had issue like that..? | 16:47 |
| jamesdenton | which in that case, the F5 healthceck can be modified | 16:48 |
| tux__ | let me check what 9200 doing in galera | 16:48 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_masakari master: Basic implementation of masakari-monitors https://review.openstack.org/584629 | 16:48 |
| tux__ | jamesdenton: here you go this is interesting... http://paste.openstack.org/show/726761/ | 16:49 |
| tux__ | so we do have mysqlcheck running on port 9200 | 16:49 |
| tux__ | Script to make a proxy (ie HAProxy) capable of monitoring Percona XtraDB Cluster nodes properly | 16:50 |
| jamesdenton | so, it looks like it returns a 503 if its out of sync? In which case you can setup an HTTP monitor on the F5. Pass if 200 fail if anything else (inc 503) | 16:51 |
| jamesdenton | and the monitor can be configured to check port 9200 | 16:51 |
| tux__ | yup!! currently your script using 9200 for ELK "create ltm monitor tcp /RPC/RPC_MON_TCP_ELASTICSEARCH { defaults-from tcp destination *:9200 }" | 16:52 |
| tux__ | jamesdenton: do you have any plan to fix your script to match current changes "f5-config.py" ? | 16:53 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_magnum master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584956 | 16:53 |
| jamesdenton | it needs to be an http monitor, not tcp | 16:54 |
| jamesdenton | and yes, eventually that will be fixed | 16:55 |
| jamesdenton | oh yeah, 9200 is used for ELK but it can also be used for galera in this case. it's not for the vip but only for monitor | 16:55 |
| tux__ | Yes that is right | 16:55 |
| jamesdenton | It would look something like: "create ltm monitor MON_HTTP_GALERA { defaults-from http destination *:9200 recv "200 OK" send "HEAD /v3 HTTP/1.1\r\nHost: rpc }" | 16:56 |
| jamesdenton | well, not /3 | 16:57 |
| jamesdenton | i mean, not /v3, just / | 16:57 |
| *** hachi has joined #openstack-ansible | 16:57 | |
| tux__ | "create ltm monitor MON_HTTP_GALERA { defaults-from http destination *:9200 recv "200 OK" send "HEAD / HTTP/1.1\r\nHost: rpc }" | 16:58 |
| tux__ | jamesdenton: ^^ | 16:58 |
| *** throwsb1 has joined #openstack-ansible | 16:58 | |
| jamesdenton | yup | 16:59 |
| tux__ | I will give it a try.. but question is how do i test this ? | 16:59 |
| jamesdenton | good question. i mean, you can check that all three are UP and reporting 200 OK | 17:00 |
| jamesdenton | if the monitor sees anything other than a 200 OK, like a 503, or there's no tcp connection, then it will fail | 17:00 |
| openstackgerrit | David Wilde proposed openstack/openstack-ansible-ops master: Move HMAC token generation to deploy_osa https://review.openstack.org/586622 | 17:01 |
| *** FrankZhang has quit IRC | 17:01 | |
| tux__ | I may kill 9200 port on one of galera node and see if F5 remove them from pool | 17:02 |
| tux__ | Oh! its shell script i can change return code to 503 and then F5 should remote that from pool.. | 17:04 |
| jamesdenton | there you go | 17:04 |
| tux__ | anyway i will test that my way... | 17:04 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Convert logstash groks to a multi-pipeline setup https://review.openstack.org/586419 | 17:06 |
| cloudnull | jrosser ^ if you have a moment and are feeling adventurous I'd love your feedback on that. | 17:06 |
| jrosser | cloudnull: might try that later :) saw it earlier and it looks quite epic | 17:07 |
| cloudnull | locally testing the mutl-pipeline setup has been great for data ingestion speed . | 17:07 |
| cloudnull | also fingreprinting has greatly reduced stored messages from (file|journal)beat | 17:08 |
| jrosser | one of my kibana monitoring screens was showing a small rate of "input errors" and retrys | 17:08 |
| tux__ | jamesdenton: Do you guys monitor OSA from host machine or do you have some kind of agent running in each container to check health ? | 17:08 |
| cloudnull | like when a syslog message repeates 70k times :) | 17:08 |
| jamesdenton | tux__ there are agents located on each node/container i believe | 17:08 |
| tux__ | jamesdenton: ah! so you guys monitor container as full machine like CPU/memory/disk or just running service? | 17:09 |
| jamesdenton | i'm not really sure, sorry. | 17:10 |
| jrosser | cloudnull: it was in the new xpack beats monitoring, "output errors" bottom right chart | 17:10 |
| jrosser | about 20/second in the "fail rates" and 1/second in the "output errors" | 17:10 |
| tux__ | jamesdenton: no worry i am looking for monitoring tool so curious what people doing out side world, anyway let me do google :) | 17:11 |
| tux__ | jamesdenton: thanks for helping me with F5 | 17:11 |
| jamesdenton | sure thing! | 17:11 |
| openstackgerrit | David Wilde proposed openstack/openstack-ansible-ops master: Deploy ELK in MNAIO https://review.openstack.org/586612 | 17:12 |
| *** vnogin has joined #openstack-ansible | 17:14 | |
| jproulx | is there an easy place to specify custom repository sources for repo container(s) to clone from? my grep'ing and googling apparently isn't what it aught to be today | 17:18 |
| cloudnull | jrosser I wonder if that was congestion or if one of the logstash nodes had some hiccups ? | 17:19 |
| jrosser | cloudnull: not sure, it's continuous | 17:19 |
| cloudnull | from a single beat? | 17:20 |
| noonedeadpunk | is it possible to get ansible 2.5+ with OSA in queens release? | 17:21 |
| jrosser | cloudnull: oh interesting it's auditbeat, all of it | 17:22 |
| odyssey4me | noonedeadpunk: it's not advisable, we pin here: https://github.com/openstack/openstack-ansible/blob/stable/queens/scripts/bootstrap-ansible.sh#L25 | 17:22 |
| jrosser | i can see that on the beats -> instances screen and sort the table by "output errors" | 17:22 |
| odyssey4me | generall we pin to an ansible major.minor per series | 17:22 |
| odyssey4me | ansible changes a bit too much between minors for it to be safe to update, and it's a lot of work most times to update | 17:23 |
| * jproulx finds the thing to stick in user_variables.yml | 17:25 | |
| noonedeadpunk | odyssey4me: the point is, that new way of service setup (actually os_keystone_endpoint module) is available only since 2.5/ | 17:27 |
| odyssey4me | noonedeadpunk: yep, if you need masakari in queens, you'll have to maintain a fork and change your fork accordingly | 17:27 |
| noonedeadpunk | odyssey4me: but yeah, ansible changes a lot. I wrote my first playbook on 0.6 version, than I wasn't writing playbook for a while (till 1.8), and now it's like a new world for me... | 17:28 |
| odyssey4me | evrardjp d34dh0r53 so, I think I've figured out the problem for why dragonflow is popping into that barbican patch.. nothing to do with the patch | 17:29 |
| *** savvas has quit IRC | 17:29 | |
| jrosser | odyssey4me: is there a way to test bionic integrated repo yet? | 17:30 |
| noonedeadpunk | odyssey4me: ok, got it. I hope that we'll upgrade to rocky one day... | 17:30 |
| odyssey4me | evrardjp d34dh0r53 as it turns out, the py_pkgs plugin is somehow not obeying the precedence of integrated repo vars over /etc/ansible/roles vars... it does a second pass to the role vars and because there's no neutron_git_project_group var there, it defaults to 'all' | 17:30 |
| odyssey4me | I have no idea why it is that it's not consistent across all roles though | 17:31 |
| odyssey4me | jrosser: nope, afaik most roles aren't even prepped yet | 17:31 |
| jrosser | odyssey4me: ok i was just taking a bit of a look at what i could get going | 17:32 |
| cloudnull | jrosser I wonder if that error with auditbeat is related to some of the recent changes? | 17:33 |
| cloudnull | or if it just needs a restart | 17:33 |
| cloudnull | you should be able to see the nodes the errors are coming from | 17:33 |
| jrosser | cloudnull: i think it's all of them | 17:33 |
| cloudnull | if its all nodes, then its likely something is very wrong | 17:33 |
| jrosser | it's likley very obvious in the log i guess | 17:34 |
| noonedeadpunk | odyssey4me: run_once was replaced back to the comparison, as we're not sure if masakari-api will be the first in the inventory? | 17:34 |
| *** gillesMo has quit IRC | 17:34 | |
| odyssey4me | jrosser: well, I guess you could disable this check and go for it: https://github.com/openstack/openstack-ansible/blob/master/playbooks/openstack-hosts-setup.yml#L50 | 17:35 |
| odyssey4me | I think evrardjp put up a review for a flag to disable it via var - but I don't think it's merged yet | 17:36 |
| odyssey4me | noonedeadpunk: yeah, if you're touching a bunch of compute nodes then those tasks should rather run on the control plane so that it's setup before the compute hosts are hit | 17:36 |
| *** hachi has quit IRC | 17:36 | |
| odyssey4me | it's similar to the other main roles | 17:36 |
| *** vnogin has quit IRC | 17:39 | |
| jrosser | cloudnull: auditbeat doesnt seem to be writing a log file - would you expect one? | 17:43 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Correct pkg_locations precedence https://review.openstack.org/586628 | 17:43 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_octavia master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584979 | 17:44 |
| lbragstad | random question - but does an aio build a clouds.yaml anywhere? | 17:45 |
| odyssey4me | lbragstad: in master right now, yes - in the home directory of the user running the playbooks | 17:45 |
| odyssey4me | ~/.config/openstack/clouds.yaml | 17:46 |
| lbragstad | on the deployment host? | 17:46 |
| odyssey4me | d34dh0r53 FrankZhang evrardjp :o bad bug: https://review.openstack.org/586628 | 17:46 |
| odyssey4me | lbragstad: yes | 17:46 |
| lbragstad | huh - yep.. i wasn't looking in /root/.config | 17:47 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Add aio host conf for barbican role https://review.openstack.org/579284 | 17:48 |
| noonedeadpunk | odyssey4me: how did you tell zuul that compute1 shouldn't be a container? | 17:48 |
| odyssey4me | noonedeadpunk: it's not in the 'all_containers' group | 17:49 |
| odyssey4me | I'm out for the w/end folks. Enough computering for today. Catch you all next week! | 17:49 |
| lbragstad | enjoy your weekend odyssey4me | 17:49 |
| noonedeadpunk | odyssey4me: have a good weekends. thanks for help! | 17:49 |
| odyssey4me | noonedeadpunk: btw I don't think that env.d file is right - there will be group overlaps due to https://bugs.launchpad.net/openstack-ansible/+bug/1623584 | 17:50 |
| openstack | Launchpad bug 1623584 in openstack-ansible "Dynamic inventory clash between host group name and container group name" [Low,Confirmed] - Assigned to Nolan Brubaker (nolan-brubaker) | 17:50 |
| odyssey4me | haven't confirmed though - not that important immediately, but I definitely do not suggest using that as-is in production | 17:50 |
| noonedeadpunk | odyssey4me: this env.d seemed to work for me, but I will double-check | 17:52 |
| noonedeadpunk | thank you for pointing me to this | 17:55 |
| noonedeadpunk | odyssey4me and sorry for disturbing you with such big change. I honestly wanted to help | 17:56 |
| tux__ | Folks if i am adding new osd-node (server) in running ceph cluster in that case should i just need to run "openstack-ansible ceph-install.yml" right? | 17:59 |
| tux__ | or i should use --limit <new-osd-node> ? | 17:59 |
| *** savvas has joined #openstack-ansible | 18:02 | |
| *** mmercer has joined #openstack-ansible | 18:02 | |
| *** cshen has joined #openstack-ansible | 18:03 | |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_sahara master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584989 | 18:10 |
| *** vollman has joined #openstack-ansible | 18:15 | |
| *** FrankZhang has joined #openstack-ansible | 18:19 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_keystone master: [TEST] bionic https://review.openstack.org/586635 | 18:19 |
| bgmccollum | i think i figured out the multipath issues i was having | 18:27 |
| *** hachi has joined #openstack-ansible | 18:29 | |
| *** rmart04 has joined #openstack-ansible | 18:31 | |
| lbragstad | i have another random aio question, i just deployed queens with 17.0.7 and i have a home network i'd like neutron to use it for dhcp so that my instances are on my home network | 18:45 |
| lbragstad | can i just update the network osa laid down or do i have to reconfigure neutron a bit? | 18:46 |
| bgmccollum | if multipath-tools is installed (which is now the default), then nova.conf *MUST* have `iscsi_use_multipath = True` set in nova.conf, or attachments won't work. If your compute hosts are also volume hosts (meaning, multipath is installed and running), then you *MUST* have `use_multipath_for_image_xfer = True` set in cinder.conf under [lvm], or volume migration won't work. | 18:47 |
| jamesdenton | lbragstad Well, neutron is gonna want to assign the address whether or not dhcp==true|false. You can disable port security on the network and let your home dhcp server give it an addr, but the IP in openstack won't match | 18:50 |
| jamesdenton | bgmccollum was recently working on something similiar | 18:50 |
| *** rmart04 has quit IRC | 18:51 | |
| openstackgerrit | David Wilde proposed openstack/openstack-ansible-ops master: Deploy ELK in MNAIO https://review.openstack.org/586612 | 18:53 |
| *** rmart04 has joined #openstack-ansible | 18:53 | |
| bgmccollum | im my home lab, i have a flat neutron network that uses the same subnet as my home network. i changed my home DHCP to exclude a certain range (10.0.1.200-10.0.1.219), and set that range as the allocation pool for that flat neutron network's subnet. this subnet also has DHCP disabled. if you pop a config drive in instance with a newish cloud-init, it will statically configure the network of the instance. | 18:54 |
| *** rmart04 has quit IRC | 18:54 | |
| bgmccollum | lbragstad ^ | 18:55 |
| *** v1a4 has joined #openstack-ansible | 18:55 | |
| bgmccollum | you might need to create an additional veth-pair, with one end in the br-vlan bridge, and the other to be used as a physical interface for neutron. | 18:59 |
| jamesdenton | where's your ipam driver bgmccollum? do it right | 19:01 |
| * bgmccollum eyeroll | 19:01 | |
| lbragstad | huh - interesting | 19:03 |
| lbragstad | i just did a basic aio install, which gives me a public (172.) and private (192.) network | 19:04 |
| jamesdenton | you should be able to create a router with that, and IIRC correctly it will snat out of the AIO | 19:05 |
| bgmccollum | did you run tempest? i don't recall the AIO creating any neutron networks by default...unless thats recently changed. | 19:05 |
| lbragstad | i'm using osa @ tag 17.0.7 | 19:06 |
| bgmccollum | yeah i haven't queened yet | 19:06 |
| tux__ | Folk, I was using OSA 16.0.x and now i have upgraded to 17.0.7 (because i want ceph-ansible 3.1 ) looking at here http://paste.openstack.org/show/726762/ | 19:07 |
| tux__ | I am seeing /etc/ansible/roles why i am still seeing old ceph-* playbooks which is v3.0.9 ? | 19:07 |
| tux__ | How do i tell OSA use ceph-ansible playbook not ceph-mgr, ceph-mon, ceph-osd | 19:08 |
| tux__ | cloudnull: ^^ | 19:09 |
| lbragstad | jamesdenton: i have one router already - which osa must have created for me | 19:09 |
| jamesdenton | you can delete all of that if you want to, but the plumbing setup isn't really conducive to the connectivity you seek. You'll probably have to massage it | 19:10 |
| lbragstad | yeah - that makes sense | 19:11 |
| lbragstad | i'm still unsure if it's possible to setup a router to use an external dhcp server, or if i just have my router do that after the fact? | 19:13 |
| jamesdenton | Well, there are ways to make an external dhcp server service nova instances... you just have to disable port security and be in the same vlan as the dhcp server | 19:13 |
| tux__ | cloudnull: why queen ceph-install.yml still using older version of ceph role http://paste.openstack.org/show/726764/ | 19:14 |
| jamesdenton | and disable neutron dhcp on the subnet | 19:14 |
| jamesdenton | but you will lose metadata in that case | 19:14 |
| jamesdenton | but you have config-drive i suppose | 19:14 |
| openstackgerrit | David Wilde proposed openstack/openstack-ansible-ops master: Deploy ELK in MNAIO https://review.openstack.org/586612 | 19:32 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_trove master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/584995 | 19:34 |
| *** lbragstad_ has joined #openstack-ansible | 19:40 | |
| *** lbragstad has quit IRC | 19:41 | |
| jrosser | bionic keystone -> green :) https://review.openstack.org/#/c/586635/ | 19:41 |
| logan- | nice jrosser! | 19:45 |
| jrosser | logan-: i'm just going to fire in a whole raft of similar for some other role tests | 19:46 |
| jrosser | see what sticks | 19:46 |
| logan- | do we have bionic jobs in the rabbit role yet | 19:47 |
| logan- | as you find this stuff maybe we should try to get all of the infra roles bionic jobs voting so we can try to start keeping everything green | 19:47 |
| jrosser | i'm not sure there is one in the rabbit role | 19:47 |
| jrosser | however thats the first problematic one | 19:47 |
| jrosser | oh i lie, yes there is | 19:48 |
| logan- | yea galera server is broken https://review.openstack.org/#/c/584609/.. it only works in the openstack gate :P | 19:48 |
| jrosser | this should fix rabbit https://review.openstack.org/#/c/586558/ | 19:49 |
| logan- | i wonder why upgrade is the only job running there | 19:50 |
| logan- | oh the others had a post failure | 19:50 |
| logan- | because of the zuul issue earlier | 19:50 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_aodh stable/queens: [TEST] bionic https://review.openstack.org/586648 | 19:54 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_barbican stable/queens: [TEST] bionic https://review.openstack.org/586649 | 19:54 |
| *** fatdragon has joined #openstack-ansible | 19:54 | |
| jrosser | hrrm. lets try that again | 19:55 |
| *** lbragstad_ is now known as lbragstad | 19:56 | |
| *** cshen has quit IRC | 19:57 | |
| openstackgerrit | David Wilde proposed openstack/openstack-ansible-ops master: Deploy ELK in MNAIO https://review.openstack.org/586612 | 19:59 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_tacker master: Install optional packages after venv path update https://review.openstack.org/586650 | 19:59 |
| *** throwsb1 has quit IRC | 20:02 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_aodh master: [TEST] bionic https://review.openstack.org/586654 | 20:03 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_barbican master: [TEST] bionic https://review.openstack.org/586655 | 20:03 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: [TEST] bionic https://review.openstack.org/586656 | 20:03 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_cinder master: [TEST] bionic https://review.openstack.org/586657 | 20:03 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_designate master: [TEST] bionic https://review.openstack.org/586658 | 20:03 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_glance master: [TEST] bionic https://review.openstack.org/586659 | 20:04 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_gnocchi master: [TEST] bionic https://review.openstack.org/586660 | 20:04 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_heat master: [TEST] bionic https://review.openstack.org/586661 | 20:04 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_horizon master: [TEST] bionic https://review.openstack.org/586662 | 20:04 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ironic master: [TEST] bionic https://review.openstack.org/586663 | 20:04 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_magnum master: [TEST] bionic https://review.openstack.org/586664 | 20:04 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_molteniron master: [TEST] bionic https://review.openstack.org/586665 | 20:04 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_neutron master: [TEST] bionic https://review.openstack.org/586666 | 20:04 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_nova master: [TEST] bionic https://review.openstack.org/586667 | 20:04 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_octavia master: [TEST] bionic https://review.openstack.org/586668 | 20:04 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_rally master: [TEST] bionic https://review.openstack.org/586669 | 20:04 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_sahara master: [TEST] bionic https://review.openstack.org/586670 | 20:04 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_swift master: [TEST] bionic https://review.openstack.org/586671 | 20:04 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_tacker master: [TEST] bionic https://review.openstack.org/586672 | 20:05 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_tempest master: [TEST] bionic https://review.openstack.org/586673 | 20:05 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_trove master: [TEST] bionic https://review.openstack.org/586674 | 20:05 |
| *** aludwar1 has joined #openstack-ansible | 20:06 | |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_barbican master: Setup oslo.messaging extra packages for optional drivers https://review.openstack.org/586675 | 20:07 |
| *** aludwar1 has quit IRC | 20:07 | |
| *** taseer1 has joined #openstack-ansible | 20:07 | |
| *** taseer2 has joined #openstack-ansible | 20:09 | |
| *** Taseer has quit IRC | 20:09 | |
| *** taseer1 has quit IRC | 20:12 | |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-os_masakari master: Install optional packages after venv path update https://review.openstack.org/586677 | 20:13 |
| *** ansmith has quit IRC | 20:19 | |
| *** taseer3 has joined #openstack-ansible | 20:25 | |
| *** taseer2 has quit IRC | 20:27 | |
| *** taseer4 has joined #openstack-ansible | 20:27 | |
| *** savvas has quit IRC | 20:28 | |
| *** savvas has joined #openstack-ansible | 20:28 | |
| *** dmsimard has quit IRC | 20:29 | |
| *** savvas has quit IRC | 20:30 | |
| *** savvas has joined #openstack-ansible | 20:30 | |
| *** taseer3 has quit IRC | 20:30 | |
| openstackgerrit | David Wilde proposed openstack/openstack-ansible-ops master: Deploy ELK in MNAIO https://review.openstack.org/586612 | 20:32 |
| *** v1a4 has quit IRC | 20:40 | |
| *** cshen has joined #openstack-ansible | 20:43 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_gnocchi master: Bring gather vars in line with other OSA roles https://review.openstack.org/586685 | 20:45 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_gnocchi master: [TEST] bionic https://review.openstack.org/586660 | 20:47 |
| *** tux__ has quit IRC | 20:50 | |
| *** hachi has quit IRC | 20:55 | |
| *** gkadam has quit IRC | 20:58 | |
| *** SimAloo has quit IRC | 20:59 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: [TEST] bionic https://review.openstack.org/586656 | 21:10 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: Add vars for Bionic https://review.openstack.org/586689 | 21:10 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_cinder master: [TEST] bionic https://review.openstack.org/586657 | 21:12 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_cinder master: Add vars for Bionic https://review.openstack.org/586690 | 21:12 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_designate master: [TEST] bionic https://review.openstack.org/586658 | 21:14 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_designate master: Add vars for Bionic https://review.openstack.org/586691 | 21:14 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_glance master: [TEST] bionic https://review.openstack.org/586659 | 21:16 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_glance master: Add vars for Bionic https://review.openstack.org/586692 | 21:16 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_gnocchi master: [TEST] bionic https://review.openstack.org/586660 | 21:17 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_gnocchi master: Add vars for Bionic https://review.openstack.org/586693 | 21:17 |
| *** hachi has joined #openstack-ansible | 21:18 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_heat master: [TEST] bionic https://review.openstack.org/586661 | 21:19 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_heat master: Add vars for Bionic https://review.openstack.org/586696 | 21:19 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ironic master: [TEST] bionic https://review.openstack.org/586663 | 21:21 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_ironic master: Add vars for Bionic https://review.openstack.org/586698 | 21:21 |
| *** cshen has quit IRC | 21:25 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_molteniron master: [TEST] bionic https://review.openstack.org/586665 | 21:25 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_molteniron master: Add vars for Bionic https://review.openstack.org/586701 | 21:25 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_neutron master: [TEST] bionic https://review.openstack.org/586666 | 21:29 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_neutron master: Add vars for Bionic https://review.openstack.org/586702 | 21:29 |
| *** noonedeadpunk has quit IRC | 21:30 | |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_nova master: [TEST] bionic https://review.openstack.org/586667 | 21:30 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_nova master: Add vars for Bionic https://review.openstack.org/586704 | 21:30 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_octavia master: [TEST] bionic https://review.openstack.org/586668 | 21:32 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_octavia master: Add vars for Bionic https://review.openstack.org/586705 | 21:32 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_sahara master: [TEST] bionic https://review.openstack.org/586670 | 21:34 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_sahara master: Add vars for Bionic https://review.openstack.org/586707 | 21:34 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_tacker master: [TEST] bionic https://review.openstack.org/586672 | 21:36 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_tacker master: Add vars for Bionic https://review.openstack.org/586708 | 21:36 |
| mnaser | jrosser: putting in work 💪 | 21:36 |
| jrosser | mnaser: :) gut tells me bionic is close..... | 21:37 |
| mnaser | woo | 21:37 |
| mnaser | thank you. | 21:37 |
| *** kberger has quit IRC | 21:38 | |
| jrosser | i've onlt addressed whats in a-r-r.txt to start with - theres probably a ton more | 21:38 |
| jrosser | *only | 21:38 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_trove master: [TEST] bionic https://review.openstack.org/586674 | 21:39 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_trove master: Add vars for Bionic https://review.openstack.org/586711 | 21:39 |
| jrosser | mnaser: thats it for today :) if you see anything obvious come out of those dive in! | 21:40 |
| *** ansmith has joined #openstack-ansible | 21:49 | |
| *** cshen has joined #openstack-ansible | 21:50 | |
| *** Darcidride has quit IRC | 21:56 | |
| *** savvas has quit IRC | 22:01 | |
| *** savvas has joined #openstack-ansible | 22:11 | |
| *** taseer has joined #openstack-ansible | 22:16 | |
| *** taseer4 has quit IRC | 22:16 | |
| *** savvas has quit IRC | 22:16 | |
| *** savvas has joined #openstack-ansible | 22:20 | |
| *** savvas has quit IRC | 22:25 | |
| *** hachi has quit IRC | 22:27 | |
| *** savvas has joined #openstack-ansible | 22:38 | |
| *** vnogin has joined #openstack-ansible | 22:40 | |
| *** mattinator has joined #openstack-ansible | 22:40 | |
| *** vnogin has quit IRC | 22:44 | |
| *** tux_ has joined #openstack-ansible | 22:45 | |
| *** tux_ has quit IRC | 22:50 | |
| *** joabdearaujo has quit IRC | 23:16 | |
| *** cshen has quit IRC | 23:38 | |
| *** cshen has joined #openstack-ansible | 23:50 | |
| *** joabdearaujo has joined #openstack-ansible | 23:50 | |
| *** cshen has quit IRC | 23:54 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!