| openstackgerrit | Jimmy McCrory proposed openstack/openstack-ansible master: Revert "Change haproxy http-check expect for nova_console" https://review.opendev.org/673929 | 00:02 |
|---|---|---|
| *** ivve has quit IRC | 00:24 | |
| *** kplant has quit IRC | 00:46 | |
| *** radeks_ has joined #openstack-ansible | 01:19 | |
| *** radeks_ has quit IRC | 01:24 | |
| *** gyee has quit IRC | 02:00 | |
| openstackgerrit | pengyuesheng proposed openstack/openstack-ansible-os_neutron master: Bump the openstackdocstheme extension to 1.20 https://review.opendev.org/672867 | 02:30 |
| openstackgerrit | pengyuesheng proposed openstack/openstack-ansible-os_octavia master: Bump the openstackdocstheme extension to 1.20 https://review.opendev.org/672869 | 02:31 |
| *** BjoernT has joined #openstack-ansible | 02:34 | |
| openstackgerrit | pengyuesheng proposed openstack/ansible-hardening master: Bump the openstackdocstheme extension to 1.20 https://review.opendev.org/673769 | 02:44 |
| openstackgerrit | pengyuesheng proposed openstack/ansible-config_template master: Bump the openstackdocstheme extension to 1.20 https://review.opendev.org/673772 | 02:45 |
| *** gkadam has joined #openstack-ansible | 03:29 | |
| *** gkadam has quit IRC | 03:34 | |
| *** gkadam has joined #openstack-ansible | 03:35 | |
| *** udesale has joined #openstack-ansible | 03:46 | |
| *** BjoernT has quit IRC | 04:43 | |
| *** raukadah is now known as chandankumar | 04:59 | |
| *** goldyfruit has joined #openstack-ansible | 04:59 | |
| *** goldyfruit has quit IRC | 05:04 | |
| *** kopecmartin|off is now known as kopecmartin | 05:46 | |
| *** radeks has joined #openstack-ansible | 05:52 | |
| *** radeks has quit IRC | 06:08 | |
| *** andrea15 has quit IRC | 06:19 | |
| *** cshen has joined #openstack-ansible | 06:40 | |
| *** andrea15 has joined #openstack-ansible | 06:44 | |
| *** pcaruana has quit IRC | 07:12 | |
| *** rpittau|afk is now known as rpittau | 07:13 | |
| *** fnpanic has joined #openstack-ansible | 07:14 | |
| fnpanic | Hello everyone | 07:14 |
| fnpanic | we are trying to add a new compute node to an existing OSA 18.1.4. the playbooks run fine without error. The nova-compute on the new hosts shows this message. | 07:20 |
| fnpanic | 2019-07-31 15:00:22.675 5732 WARNING nova.compute.manager [req-d06045c1-68fd-4d4d-8978-1605964a298b - - - - -] No compute node record found for host tcpuko3. If this is the first time this service is starting on this host, then you can ignore this warning.: ComputeHostNotFound_Remote: Compute host newnode3 could not be found. | 07:20 |
| fnpanic | this shows up everytime we start the service | 07:21 |
| fnpanic | neutron agents run fine on the node. also the nova service is there but shows down and the host is not in the hypervisor list | 07:30 |
| cshen | fnpanic: which playbook of OSA have you run? | 07:40 |
| *** shyamb has joined #openstack-ansible | 07:42 | |
| *** pcaruana has joined #openstack-ansible | 07:42 | |
| *** jsquare has quit IRC | 07:44 | |
| masterpe | cshen: we followed https://docs.openstack.org/openstack-ansible/rocky/admin/scale-environment.html#add-a-compute-host | 07:48 |
| *** jsquare has joined #openstack-ansible | 07:49 | |
| masterpe | After doing some network test, we found out that we have some MTU issues to that node | 07:49 |
| masterpe | We are going to solve that first and see if that solves the issue | 07:50 |
| *** ivve has joined #openstack-ansible | 07:51 | |
| *** tosky has joined #openstack-ansible | 07:54 | |
| *** electrofelix has joined #openstack-ansible | 07:58 | |
| *** goldyfruit has joined #openstack-ansible | 08:04 | |
| cshen | masterpe: ok, we followed the same page, and it worked. | 08:06 |
| *** goldyfruit has quit IRC | 08:09 | |
| *** ivve has quit IRC | 08:30 | |
| *** ivve has joined #openstack-ansible | 08:40 | |
| *** shyamb has quit IRC | 09:06 | |
| *** shyamb has joined #openstack-ansible | 09:07 | |
| *** shyamb has quit IRC | 09:12 | |
| *** ivve has quit IRC | 09:37 | |
| *** goldyfruit has joined #openstack-ansible | 09:55 | |
| *** shyamb has joined #openstack-ansible | 09:55 | |
| *** goldyfruit has quit IRC | 10:00 | |
| *** prometheanfire has quit IRC | 10:04 | |
| *** rpittau is now known as rpittau|bbl | 10:04 | |
| *** fnpanic has quit IRC | 10:08 | |
| *** prometheanfire has joined #openstack-ansible | 10:50 | |
| *** cshen has quit IRC | 11:03 | |
| *** ivve has joined #openstack-ansible | 11:07 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_heat master: Convert dynamic includes to static imports https://review.opendev.org/674003 | 11:09 |
| *** goldyfruit has joined #openstack-ansible | 11:14 | |
| *** cshen has joined #openstack-ansible | 11:14 | |
| masterpe | cshen: thanks for the conformation. | 11:19 |
| *** cshen has quit IRC | 11:19 | |
| *** shyamb has quit IRC | 11:20 | |
| *** cshen has joined #openstack-ansible | 11:21 | |
| *** kplant has joined #openstack-ansible | 11:21 | |
| *** goldyfruit has quit IRC | 11:21 | |
| *** shyamb has joined #openstack-ansible | 11:21 | |
| masterpe | cshen: we are missing the ssh key in /var/lib/nova/.ssh/authorized_keys | 11:21 |
| masterpe | so live-migration is failing | 11:22 |
| *** ivve has quit IRC | 11:37 | |
| CeeMac | masterpe: I seem to recall having an issue with some of the nova key related tags not working | 11:37 |
| CeeMac | I went with the sledge hammer approach of regenerating the keys, but I wasnt in production at the time | 11:38 |
| masterpe | :) | 11:38 |
| *** shyamb has quit IRC | 11:46 | |
| *** shyamb has joined #openstack-ansible | 11:47 | |
| *** udesale has quit IRC | 11:49 | |
| *** udesale has joined #openstack-ansible | 11:50 | |
| jrosser | CeeMac: I keep meaning to check if running the nova role with —limit messes up the key collection/distribution | 11:53 |
| CeeMac | jrosser: it definitely seems a possibility | 11:54 |
| CeeMac | i seem to recall the nova-config tag didn't pull properly either for some important tasks as the tasks were bundled in post-install which doesn't play when nova-config tag is used | 11:55 |
| CeeMac | i'll be needing to add a couple of new hosts in a few weeks so I'll most likely find out :) | 11:56 |
| jrosser | Submit bugs or fixes :) | 11:57 |
| CeeMac | maybe :) | 11:57 |
| CeeMac | if i find something I'll take a look see if i can untangle it | 11:58 |
| CeeMac | i think i still have an open bug for the nova-config tag actually | 11:59 |
| CeeMac | i do, https://bugs.launchpad.net/openstack-ansible/+bug/1815043 | 12:00 |
| openstack | Launchpad bug 1815043 in openstack-ansible "running os-nova-install.yml --tags nova-config does note generate config" [Undecided,New] | 12:00 |
| *** ansmith has quit IRC | 12:00 | |
| *** jcosmao has joined #openstack-ansible | 12:04 | |
| *** gkadam has quit IRC | 12:10 | |
| *** omegapoint has joined #openstack-ansible | 12:17 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_heat master: service_setup: refactor service setup to a single file https://review.opendev.org/673778 | 12:19 |
| *** gillesMo has joined #openstack-ansible | 12:42 | |
| *** ansmith has joined #openstack-ansible | 12:48 | |
| *** rpittau|bbl is now known as rpittau | 12:57 | |
| *** tosky has quit IRC | 13:10 | |
| *** tosky has joined #openstack-ansible | 13:10 | |
| *** shyamb has quit IRC | 13:11 | |
| *** joshualyle has quit IRC | 13:44 | |
| *** joshualyle has joined #openstack-ansible | 13:47 | |
| *** joshualyle has quit IRC | 13:52 | |
| *** joshualyle has joined #openstack-ansible | 13:54 | |
| *** joshualyle has quit IRC | 13:54 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_cloudkitty master: service_setup: refactor service setup to a single file https://review.opendev.org/673542 | 13:58 |
| *** lbragstad has joined #openstack-ansible | 13:59 | |
| *** BjoernT has joined #openstack-ansible | 14:01 | |
| *** spatel has joined #openstack-ansible | 14:06 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_cinder master: service_setup: region override https://review.opendev.org/673500 | 14:07 |
| spatel | Folks quick question, can i restart controller node one by one or do i need to do something else before that? | 14:09 |
| *** BjoernT_ has joined #openstack-ansible | 14:09 | |
| *** BjoernT has quit IRC | 14:10 | |
| noonedeadpunk | spatel: of course you can, but I'd recommend to somehow drop memcahced server on the mgmt node for maintenance from service configs (at least from keystone) | 14:10 |
| spatel | somehow drop memcahced server on the mgmt node for maintenance ?? | 14:11 |
| noonedeadpunk | Since memchached don't have HA and unavailability of one memcahced server will degrade performance of issuing tokens | 14:11 |
| spatel | one more point my cluster isn't in production yet... | 14:11 |
| noonedeadpunk | spatel: I mean drop from service configs | 14:11 |
| noonedeadpunk | Ah, than it's ok:) | 14:11 |
| spatel | noonedeadpunk: how to drop from service config? | 14:11 |
| noonedeadpunk | For example, override var memcahced_servers and launching os-keystone-setup.yml | 14:12 |
| spatel | hmmm! | 14:13 |
| noonedeadpunk | Just check that galera is ok after node goes up and before shutting down next one | 14:13 |
| spatel | noonedeadpunk: i will, i am more worried about rabbitmq because last time it messed up | 14:14 |
| spatel | let me reboot right now and see... cross my figure.. | 14:14 |
| noonedeadpunk | So generally rabbitmq shoudl be ok... | 14:14 |
| *** mkuf has joined #openstack-ansible | 14:16 | |
| *** jhesketh has quit IRC | 14:23 | |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_designate master: service_setup: refactor service setup to a single file https://review.opendev.org/673484 | 14:29 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_aodh master: service_setup: refactor service setup to a single file https://review.opendev.org/673495 | 14:31 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_blazar master: service_setup: refactor service setup to a single file https://review.opendev.org/673498 | 14:31 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_congress master: service_setup: refactor service setup to a single file https://review.opendev.org/673525 | 14:33 |
| *** mkuf_ has joined #openstack-ansible | 14:53 | |
| *** mkuf has quit IRC | 14:54 | |
| *** joshualyle has joined #openstack-ansible | 14:58 | |
| *** mkuf has joined #openstack-ansible | 14:59 | |
| *** mkuf_ has quit IRC | 14:59 | |
| *** joshualyle has quit IRC | 15:02 | |
| *** cshen has quit IRC | 15:09 | |
| *** joshualyle has joined #openstack-ansible | 15:09 | |
| *** joshualyle has quit IRC | 15:13 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_ironic master: tftp: start on boot & fix tftp paths https://review.opendev.org/674071 | 15:14 |
| omegapoint | hey everyone, i'm trying to set up a testbed for our openstack deployment, and i'm running into very strange issues where the playbooks fail to gather facts on a _completely_ fresh instance, this always happens: https://pastebin.com/LQt3qWKG ssh just hangs completely; curiously the step before (ensure python installed) works just fine | 15:25 |
| omegapoint | the testbed is just a few instances of the xenial cloud image and a network configuration based on the .test template, so putting the 3 bridges on 3 different vlan trunks on top of a single interface. | 15:27 |
| *** gyee has joined #openstack-ansible | 15:27 | |
| mnaser | is sounds like a network thing | 15:28 |
| openstackgerrit | Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_blazar master: service_setup: refactor service setup to a single file https://review.opendev.org/673498 | 15:29 |
| noonedeadpunk | omegapoint: on what osa version your fresh install is based? | 15:30 |
| noonedeadpunk | since we don't support xenial for stein... | 15:30 |
| omegapoint | @mnaser yeah i thought the same, not sure how to pinpoint what's wrong though. @noonedeadpunk still running 16.0.27, yeah it's pretty old which is why i'm setting up the testbed to prepare for some major upgrades | 15:31 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_ironic master: tftp: start on boot & fix tftp paths https://review.opendev.org/674071 | 15:35 |
| noonedeadpunk | mnaser: I've placed comment for ^ | 15:45 |
| mnaser | replied :) | 15:46 |
| openstackgerrit | Bjoern Teipel proposed openstack/openstack-ansible master: Revert ansible-playbook link to the unmodified binary https://review.opendev.org/668185 | 15:46 |
| openstackgerrit | Georgina Shippey proposed openstack/openstack-ansible-ops master: Config updates for elk 7.x https://review.opendev.org/669852 | 15:46 |
| openstackgerrit | Georgina Shippey proposed openstack/openstack-ansible-ops master: Config updates for elk 7.x https://review.opendev.org/669852 | 15:48 |
| noonedeadpunk | mnaser: but if role fails at some point doesn't it mean, that it's not functional? | 15:48 |
| noonedeadpunk | and service deployed by it not functional as well? | 15:49 |
| *** stingrayza has quit IRC | 15:50 | |
| mnaser | noonedeadpunk: i mean maybe someone hit ctrl+c or it failed *after* that task BUT the handlers didnt run for soem reason | 15:50 |
| mnaser | if you rerun the same playbook, the handler wont run, because changed=false for those tasks | 15:50 |
| noonedeadpunk | So I'd say we shouldn't really care much about situations, when role can accidentally fail before running handlers | 15:50 |
| noonedeadpunk | otherwise we should stop using handlers at all | 15:50 |
| noonedeadpunk | oh | 15:51 |
| noonedeadpunk | that's what you mean... | 15:51 |
| *** stingrayza has joined #openstack-ansible | 15:51 | |
| noonedeadpunk | But still not sure, since we can have plenty of such cases, as we didn't cover anything that way... | 15:52 |
| *** jcosmao has left #openstack-ansible | 15:53 | |
| noonedeadpunk | Ah, systemd role launches services in case of source installation... | 15:55 |
| *** gkadam has joined #openstack-ansible | 15:57 | |
| openstackgerrit | Bjoern Teipel proposed openstack/openstack-ansible master: Revert ansible-playbook link to the unmodified binary https://review.opendev.org/668185 | 15:57 |
| *** gkadam has quit IRC | 16:01 | |
| openstackgerrit | Bjoern Teipel proposed openstack/openstack-ansible master: Revert ansible-playbook link to the unmodified binary https://review.opendev.org/668185 | 16:08 |
| *** ksdean has joined #openstack-ansible | 16:08 | |
| *** ksdean has quit IRC | 16:08 | |
| *** ksdean has joined #openstack-ansible | 16:09 | |
| *** ksdean has quit IRC | 16:10 | |
| *** ksdean has joined #openstack-ansible | 16:12 | |
| *** lbragstad has quit IRC | 16:29 | |
| jrosser | oh dear download.cirros-cloud.net looks dead | 16:35 |
| jrosser | mnaser: ^ perhaps we really do need to fork cirros | 16:37 |
| mnaser | oh no | 16:38 |
| mnaser | loads for me jrosser | 16:38 |
| jrosser | oh right it pings again | 16:38 |
| jrosser | we just had a few jobs fail to d/l the image | 16:38 |
| jrosser | and it was properly down | 16:38 |
| *** rpittau is now known as rpittau|afk | 16:40 | |
| omegapoint | @mnaser for the record, turned out to be a weird ARP problem because of janky network setup, figured it out with a tcpdump | 16:43 |
| mnaser | omegapoint: cool thats good | 16:44 |
| *** ivve has joined #openstack-ansible | 16:55 | |
| *** kdean has joined #openstack-ansible | 17:04 | |
| *** allanb has quit IRC | 17:04 | |
| *** ksdean has quit IRC | 17:06 | |
| *** electrofelix has quit IRC | 17:06 | |
| *** allanb has joined #openstack-ansible | 17:09 | |
| *** omegapoint has quit IRC | 17:09 | |
| *** allanb has quit IRC | 17:10 | |
| *** allanb has joined #openstack-ansible | 17:11 | |
| *** allanb has quit IRC | 17:12 | |
| *** allanb has joined #openstack-ansible | 17:12 | |
| *** udesale has quit IRC | 17:14 | |
| *** allanb has quit IRC | 17:15 | |
| *** allanb has joined #openstack-ansible | 17:16 | |
| *** allanb has quit IRC | 17:19 | |
| *** allanb has joined #openstack-ansible | 17:22 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Mark Stein as Released https://review.opendev.org/674096 | 17:24 |
| *** kopecmartin is now known as kopecmartin|off | 17:24 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible stable/stein: Mark Stein as released https://review.opendev.org/674098 | 17:26 |
| evrardjp | mnaser: ^ those two should be merged, else we look late, which we weren't | 17:26 |
| evrardjp | not urgent but public facing | 17:26 |
| mnaser | evrardjp: oops. /me will wait for it to build docs and will merge | 17:27 |
| mnaser | evrardjp: got a suse machine handy? | 17:37 |
| mnaser | the tftp package.. what is the service name it uses & does it default to /var/lib/tftpboot or /tftpboot .. or somethign else? | 17:37 |
| mnaser | https://review.opendev.org/#/c/674071/ fixing ironic for centos uncovered it was probably always broken in suse too | 17:37 |
| *** tosky has quit IRC | 18:01 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: config: disable notifications by default https://review.opendev.org/673882 | 20:03 |
| kaiokmo | mnaser: I think tftpd-hpa uses /var/lib/tftpboot by default | 20:07 |
| openstackgerrit | Douglas Mendizábal proposed openstack/ansible-role-thales-hsm master: Fix typo in client playbook https://review.opendev.org/674128 | 20:17 |
| *** tosky has joined #openstack-ansible | 20:17 | |
| *** ansmith has quit IRC | 20:20 | |
| spatel | what could be wrong here? - http://paste.openstack.org/show/755245/ | 20:24 |
| *** kplant has quit IRC | 20:29 | |
| jrosser | spatel: i guess some service down, and your loadbalancer roundrobin from something broken to something working? | 20:32 |
| spatel | hmmm i wonder if cinder is in zombie state... i have reboot all controller one by one and after that it started throwing that error | 20:32 |
| spatel | let me restart all cinder container and seee | 20:33 |
| jrosser | run your command a few times and see if its 1-in-N pattern that breaks | 20:33 |
| openstackgerrit | Merged openstack/ansible-role-thales-hsm master: Fix typo in client playbook https://review.opendev.org/674128 | 20:35 |
| spatel | jrosser: after restarting all cinder-api container it fixed issue.. very strange | 20:36 |
| spatel | let me verify again to see all service are up.. | 20:36 |
| *** spatel has quit IRC | 20:41 | |
| *** spatel has joined #openstack-ansible | 21:02 | |
| *** cloudnull has quit IRC | 21:06 | |
| *** cloudnull has joined #openstack-ansible | 21:07 | |
| *** spatel has quit IRC | 21:17 | |
| *** tosky has quit IRC | 22:07 | |
| *** BjoernT_ has quit IRC | 22:17 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: config: disable notifications by default https://review.opendev.org/673878 | 22:18 |
| *** jrosser_ has joined #openstack-ansible | 22:29 | |
| *** jtanner_ has joined #openstack-ansible | 22:29 | |
| *** fyx_ has joined #openstack-ansible | 22:30 | |
| *** gundalow_ has joined #openstack-ansible | 22:30 | |
| *** fyx has quit IRC | 22:37 | |
| *** gundalow has quit IRC | 22:37 | |
| *** jtanner has quit IRC | 22:37 | |
| *** janno has quit IRC | 22:37 | |
| *** jrosser has quit IRC | 22:37 | |
| *** gundalow_ is now known as gundalow | 22:37 | |
| *** fyx_ is now known as fyx | 22:37 | |
| *** jrosser_ is now known as jrosser | 22:37 | |
| *** jtanner_ is now known as jtanner | 22:37 | |
| *** rickflare has quit IRC | 22:39 | |
| *** ivve has quit IRC | 23:05 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_ironic master: redhat: add qemu-img-ev to dependencies https://review.opendev.org/673905 | 23:09 |
| *** ianychoi has quit IRC | 23:12 | |
| *** spatel has joined #openstack-ansible | 23:36 | |
| *** trident has quit IRC | 23:54 | |
| *** spatel has quit IRC | 23:58 | |
| *** trident has joined #openstack-ansible | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!