| ThiagoCMC | Somehow, the second run is faling here: "TASK [galera_server : Create galera users]", it feels like that I'm facing this: https://bugs.launchpad.net/openstack-ansible/+bug/1745281 BUT, I'm Ubuntu, not CentOS... :-( | 00:00 |
|---|---|---|
| openstack | Launchpad bug 1762421 in openstack-ansible "duplicate for #1745281 openstack-ansible-galera_server role needlessly removes and reinstalls MariaDB-server" [High,Fix released] - Assigned to Vadim Kuznetsov (vakuznet) | 00:00 |
| *** macz_ has quit IRC | 00:00 | |
| ThiagoCMC | This is really weird! My MariaDB Galera cluster is running, all servers are "Primary". | 00:06 |
| ThiagoCMC | However, the logs of the first one shows an error: "This installation of MariaDB is already upgraded to 10.4.14-MariaDB, use --force if you still need to run mysql_upgrade", then: | 00:06 |
| ThiagoCMC | ERROR 1047 (08S01) at line 1: WSREP has not yet prepared node for application use | 00:07 |
| ThiagoCMC | No idea about what's happening... | 00:07 |
| ThiagoCMC | :-( | 00:07 |
| ThiagoCMC | I'll destroy/create it. | 00:09 |
| *** gshippey has quit IRC | 00:16 | |
| ThiagoCMC | After destroying/creating the first Galera container, the galera-install.yml worked again! Very weird... | 00:21 |
| *** vesper11 has joined #openstack-ansible | 01:09 | |
| ThiagoCMC | Something really bad happened here... Somehow, the first Galera node got screwed, then, I destroyed/created it again. I was running this second OSA run... It's in a different state: http://paste.openstack.org/show/800607/ - it feels like that this new Galera node started empty, not syncing with others and then, when I executed the OSA, it's treating it as a fresh install! Services can't access the DBs | 01:40 |
| ThiagoCMC | : http://paste.openstack.org/show/800608/ | 01:40 |
| ThiagoCMC | All the Projects disappeared... All users... All volumes, all networks... | 01:41 |
| ThiagoCMC | I can still see Glance images "lost" on Ceph but, they don't show up on Horizon, neither on "openstack image list"... | 01:42 |
| ThiagoCMC | :-O | 01:42 |
| *** cshen has joined #openstack-ansible | 01:42 | |
| *** cshen has quit IRC | 01:46 | |
| *** spatel has joined #openstack-ansible | 01:48 | |
| ThiagoCMC | Galera sucks... lol | 01:49 |
| ThiagoCMC | I'm scared | 01:49 |
| *** rfolcOUT has joined #openstack-ansible | 02:50 | |
| *** rfolcOUT has quit IRC | 03:05 | |
| *** cshen has joined #openstack-ansible | 03:43 | |
| *** cshen has quit IRC | 03:47 | |
| *** stduolc has joined #openstack-ansible | 03:50 | |
| *** stduolc1 has joined #openstack-ansible | 03:52 | |
| *** stduolc has quit IRC | 03:53 | |
| *** stduolc1 has quit IRC | 03:53 | |
| *** stduolc has joined #openstack-ansible | 03:53 | |
| *** evrardjp has quit IRC | 05:33 | |
| *** evrardjp has joined #openstack-ansible | 05:33 | |
| *** cshen has joined #openstack-ansible | 05:43 | |
| *** cshen has quit IRC | 05:48 | |
| *** cshen has joined #openstack-ansible | 06:15 | |
| *** cshen has quit IRC | 06:19 | |
| *** gyee has quit IRC | 06:30 | |
| *** spatel has quit IRC | 06:37 | |
| *** pto_ has quit IRC | 06:37 | |
| *** pto has joined #openstack-ansible | 06:37 | |
| *** pto has quit IRC | 06:40 | |
| *** pto has joined #openstack-ansible | 06:40 | |
| *** mrda has quit IRC | 06:43 | |
| *** stduolc has quit IRC | 06:44 | |
| *** stduolc has joined #openstack-ansible | 06:45 | |
| *** mrda has joined #openstack-ansible | 06:48 | |
| *** SiavashSardari has joined #openstack-ansible | 06:58 | |
| *** stduolc has quit IRC | 07:00 | |
| *** stduolc has joined #openstack-ansible | 07:00 | |
| *** cshen has joined #openstack-ansible | 07:04 | |
| *** stduolc has quit IRC | 07:10 | |
| *** stduolc has joined #openstack-ansible | 07:10 | |
| *** npalladium has joined #openstack-ansible | 07:14 | |
| *** pto has quit IRC | 07:17 | |
| *** miloa has joined #openstack-ansible | 07:17 | |
| *** pto has joined #openstack-ansible | 07:17 | |
| *** pcaruana has joined #openstack-ansible | 07:29 | |
| *** pcaruana has quit IRC | 07:32 | |
| *** pcaruana has joined #openstack-ansible | 07:33 | |
| *** pcaruana has quit IRC | 07:40 | |
| stduolc | http://paste.openstack.org/show/800612/ I can't run the setup-hosts.yaml | 07:44 |
| *** pcaruana has joined #openstack-ansible | 07:51 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_adjutant master: Define condition for the first play host one time https://review.opendev.org/c/openstack/openstack-ansible-os_adjutant/+/764656 | 07:54 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Define condition for the first play host one time https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/754411 | 07:54 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_swift master: Define condition for the first play host one time https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/754428 | 07:54 |
| *** mmethot_ has joined #openstack-ansible | 08:07 | |
| *** mmethot has quit IRC | 08:09 | |
| *** cshen has quit IRC | 08:13 | |
| jrosser | ThiagoCMC: did you only have two galera nodes? | 08:26 |
| *** rpittau|afk is now known as rpittau | 08:28 | |
| *** andrewbonney has joined #openstack-ansible | 08:34 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_ironic master: Define condition for the first play host one time https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/756340 | 08:41 |
| *** cshen has joined #openstack-ansible | 08:55 | |
| *** MickyMan77 has left #openstack-ansible | 08:55 | |
| *** MickyMan77 has joined #openstack-ansible | 08:55 | |
| *** SiavashSardari has quit IRC | 08:56 | |
| *** tosky has joined #openstack-ansible | 09:05 | |
| *** SiavashSardari has joined #openstack-ansible | 09:11 | |
| *** npalladium has quit IRC | 09:11 | |
| *** luksky has joined #openstack-ansible | 09:15 | |
| *** sshnaidm is now known as sshnaidm|ruck | 09:34 | |
| *** SiavashSardari has quit IRC | 09:34 | |
| *** sshnaidm|ruck has quit IRC | 10:01 | |
| *** sshnaidm has joined #openstack-ansible | 10:01 | |
| noonedeadpunk | honestly - I'm about to mark debian as non-voting for octavia.... | 10:18 |
| noonedeadpunk | well, that's interesting https://zuul.opendev.org/t/openstack/build/493fb4d9f67a44e19d7547ffac3513f0/log/logs/host/haproxy.service.journal-00-28-19.log.txt#1618 | 10:29 |
| jrosser | oh yes thats super odd | 10:34 |
| jrosser | i wonder if there is something happens at that point like attaching amphora to lb-mgmt network and the host networking just goes crazy | 10:34 |
| noonedeadpunk | it ooms | 10:43 |
| noonedeadpunk | or that... | 10:43 |
| noonedeadpunk | (but ooms for sure according to syslog) | 10:43 |
| jrosser | oh, which thing oom? | 10:44 |
| noonedeadpunk | some uwsgi | 10:53 |
| noonedeadpunk | according to log of haproxy that should be all uwsgi.... | 10:53 |
| *** sshnaidm is now known as sshnaidm|afk | 10:55 | |
| *** SiavashSardari has joined #openstack-ansible | 11:12 | |
| *** MickyMan77 has quit IRC | 11:36 | |
| *** sshnaidm|afk is now known as sshnaidm|ruck | 11:39 | |
| *** stduolc has quit IRC | 11:43 | |
| *** stduolc has joined #openstack-ansible | 11:43 | |
| *** shyamb has joined #openstack-ansible | 11:51 | |
| SiavashSardari | hey guys, I was wondering whats is the quickest way of setup a test-bed for OSA? currently, I'm using terraform to create resources on openstack and then deploy OSA on top of them, but it takes too long. I'd like to know what other (faster) options are available. | 11:55 |
| *** rfolcOUT has joined #openstack-ansible | 12:06 | |
| *** rfolcOUT has quit IRC | 12:07 | |
| *** rfolco has joined #openstack-ansible | 12:07 | |
| *** mgariepy has quit IRC | 12:09 | |
| jrosser | SiavashSardari: some time ago a *lot* of effort was put into this https://github.com/openstack/openstack-ansible-ops/tree/master/multi-node-aio | 12:18 |
| jrosser | if you have a giant enough server that would bring up KVM vm for each of the hosts you wanted then put OSA on the top | 12:19 |
| jrosser | best suited to running on a ironic or real physical server with sufficient resources | 12:19 |
| jrosser | if you want a quick turn around then that had some bits to snapshot the VM at certain parts of the deployment | 12:20 |
| jrosser | then for most development / bug fixing it's all done in AIO which you need nothing more than an 4or8C/8G+ VM | 12:22 |
| jrosser | and those go noticeably quicker if you can back them onto local NVMe | 12:22 |
| *** pfsmorigo has left #openstack-ansible | 12:24 | |
| SiavashSardari | jrosser Thank you, unfortunately I found out about ops repo after I used terraform to do the same (well, kinda the same). I've never tested AIO, but you're right AIO would be the fastest way of testing. Thanks for the suggestion. | 12:26 |
| admin0 | hi .. the octavia_neutron_management_network_name -- this is a private range right? | 12:32 |
| admin0 | or it needs to be a public ip range ? | 12:33 |
| noonedeadpunk | I think it's private net which is eventually br-lbaas side on the neutron | 12:33 |
| admin0 | noonedeadpunk, when adding this network, add it as untagged/flat using br-lbaas as physical interface ? | 12:44 |
| guilhermesp | admin0: yeah, it should be a br-lbaas that allows amphora instances to talk to control plane ( octavia containers in case you run a lxc deployment ) | 12:44 |
| admin0 | https://developer.rackspace.com/docs/private-cloud/rpc/master/rpc-octavia-internal/octavia-install-guide/-- this skips this part . and documentation is vauge | 12:45 |
| noonedeadpunk | all depends on your networking - I'd expect it to be tagged vlan honestly but all depends. The idea here is that the same network should be prresent in neutron and inside octavia containers | 12:46 |
| noonedeadpunk | and it should be pretty big in terms of the address space | 12:47 |
| noonedeadpunk | (like /22 or /20 to be able to have all LB spawned within this netwrok. and each LB occupies min 2 ports) | 12:48 |
| openstackgerrit | Merged openstack/openstack-ansible-os_adjutant master: Reduce number of processes on small systems https://review.opendev.org/c/openstack/openstack-ansible-os_adjutant/+/764654 | 12:51 |
| *** vesper11 has quit IRC | 12:53 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Update variables for default Zun deployments and AIO https://review.opendev.org/c/openstack/openstack-ansible/+/763562 | 12:54 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Implement uWSGI for neutron-api https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/486156 | 12:55 |
| *** jbadiapa has joined #openstack-ansible | 12:55 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_swift master: Stop to use the __future__ module. https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/732883 | 12:58 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_swift master: Define condition for the first play host one time https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/754428 | 13:01 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Decrease amphora image RAM for CI https://review.opendev.org/c/openstack/openstack-ansible/+/765134 | 13:10 |
| admin0 | fatal: [c1_octavia_server_container-64db75e4 -> 10.101.73.210]: FAILED! => {"attempts": 5, "changed": false, "msg": "ConflictException: 409: Client Error for url: http://10.101.72.9:9696/v2.0/networks.json, Unable to create the flat network. Physical network lbaas is in use."} | 13:10 |
| admin0 | what does that mean ? | 13:10 |
| admin0 | the lbaas was just added .. not used by anyone else | 13:11 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Fix linter errors https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/755576 | 13:11 |
| admin0 | or please let me know how to add this network | 13:11 |
| admin0 | i need to add it on top of br-vlan .. or it has to run on top of br-lbaas ? | 13:14 |
| admin0 | br-lbaas is on top of a tagged vlan interface | 13:14 |
| admin0 | i mean its another bridge on the internal network tagged to a specific vlan | 13:15 |
| noonedeadpunk | I'd expect to see the following networking https://satishdotpatel.github.io/assets/images/2020-09-28-openstack-ansible-octavia/octavia-lbaas-network-diagram.png | 13:16 |
| *** mgariepy has joined #openstack-ansible | 13:17 | |
| noonedeadpunk | br-lbaas should net exist on computing nodes | 13:17 |
| noonedeadpunk | as neutron should create bridge and it would be vlan network then | 13:17 |
| noonedeadpunk | br-lbaas should be created only on the controllers | 13:18 |
| noonedeadpunk | you can check out https://satishdotpatel.github.io//openstack-ansible-octavia/ as well | 13:19 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Updated from OpenStack Ansible Tests https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/754177 | 13:26 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Updated from OpenStack Ansible Tests https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/754177 | 13:26 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/ussuri: Set Bridge Information for NSX Integration https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/750330 | 13:27 |
| *** SiavashSardari has quit IRC | 13:29 | |
| *** pcaruana has quit IRC | 13:39 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Drop octavia v1 api options https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/765138 | 13:46 |
| noonedeadpunk | jrosser: what do you think about https://docs.openstack.org/releasenotes/keystonemiddleware/ocata.html ? | 13:49 |
| noonedeadpunk | I think we haven't really implement that anywhere yet? | 13:50 |
| jrosser | i don't think we have | 13:51 |
| jrosser | is this in relation to the ML post about a nova/osa issue? | 13:51 |
| *** spatel has joined #openstack-ansible | 13:52 | |
| noonedeadpunk | um... nova/osa issue in ML?:) | 13:52 |
| noonedeadpunk | I was looking through warnings in logs | 13:53 |
| noonedeadpunk | `Instances can only connect to provider-net via tenant-net`? | 13:53 |
| *** spatel has quit IRC | 13:53 | |
| noonedeadpunk | that ML you meant? | 13:53 |
| *** pto has quit IRC | 13:56 | |
| jrosser | yes, that was heading off into service tokens (incorrectly i think( | 13:56 |
| *** pto has joined #openstack-ansible | 13:56 | |
| jrosser | but this is helpful https://opendev.org/openstack/keystone/src/branch/master/doc/source/admin/manage-services.rst | 13:56 |
| *** pcaruana has joined #openstack-ansible | 13:56 | |
| noonedeadpunk | yeah, I think it's high time we implemented this tbh.... | 13:57 |
| noonedeadpunk | but not sure if I shuould do it for V lol | 13:57 |
| noonedeadpunk | I was just going to somehow merge octavia stuff and branch finally | 13:58 |
| noonedeadpunk | but considering it's deprecated in O lol | 13:58 |
| *** spatel has joined #openstack-ansible | 13:59 | |
| jrosser | hmm? | 14:00 |
| jrosser | you mean something got missed very long ago? | 14:00 |
| noonedeadpunk | yeah, we are supposed to use service tokens since O as it was deprecated back there | 14:01 |
| noonedeadpunk | `For backwards compatibility the service_token_roles_required option in [keystone_authtoken] was added. The option defaults to False and has been immediately deprecated. This will allow the current behaviour that service tokens are validated but not checked for roles to continue. The option should be set to True as soon as possible. ` | 14:01 |
| noonedeadpunk | Reno from OCata | 14:01 |
| jrosser | this might be super tricky to merge | 14:02 |
| jrosser | as the services kind of all talk to each other | 14:02 |
| *** pto has quit IRC | 14:07 | |
| ThiagoCMC | jrosser, nop, I have 5 Galera nodes (controllers). Well, I had, I re-deployed it again, from scratch. It's very hard to actually maintain a deployment alive and healthy! | 14:14 |
| *** shyamb has quit IRC | 14:17 | |
| jrosser | ThiagoCMC: the galera heathcheck from haproxy should exclude db instances that are not joined to the cluster | 14:18 |
| jrosser | theres a quote complicated piece of testing to ensure that never happens | 14:19 |
| ThiagoCMC | Hmm... I see... Well, something happened! I wasn't able to login to OpenStack, tried a re-run and something worse took place. I decided to re-deploy (this is my personal lab anyway but, I want it to be my personal prod env). | 14:20 |
| ThiagoCMC | No worries... I'm just to get more and more experience with all the pieces. ^_^ | 14:21 |
| noonedeadpunk | jrosser: another annoying thing - json policy being deprecated in V and dropped in W | 14:21 |
| jrosser | ThiagoCMC: https://docs.openstack.org/openstack-ansible/rocky/admin/maintenance-tasks/galera.html | 14:23 |
| ThiagoCMC | Thanks! | 14:24 |
| *** cshen has quit IRC | 14:24 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Drop octavia v1 api options https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/765138 | 14:27 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Cleanup octavia configuration https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/765148 | 14:31 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Cleanup octavia configuration https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/765148 | 14:34 |
| *** OliverW has joined #openstack-ansible | 14:41 | |
| *** klamath_atx has quit IRC | 14:44 | |
| *** nurdie has joined #openstack-ansible | 14:44 | |
| OliverW | Hi! I asked a question on the openstack-discuss mailing list and was asked to join this channel. I can start VMs connected to a provider network via a virtual network but not when connected directly to a provider network. (OSA Ussuri) | 14:46 |
| noonedeadpunk | hey | 14:46 |
| noonedeadpunk | yeah it's just painfully to read logs there ( | 14:47 |
| noonedeadpunk | ok, let's revise what has been found out. | 14:50 |
| noonedeadpunk | I suppose you don't have any policy overrides placed? | 14:50 |
| noonedeadpunk | and can you remid if anything you've found in journalctl for linuxbridge agent on the compute hosts? | 14:52 |
| OliverW | overrides like in /etc/openstack_deploy/user_variables.yml ? | 14:52 |
| noonedeadpunk | yep | 14:52 |
| noonedeadpunk | as it feels smth like not present flat interface on the compute | 14:52 |
| OliverW | i do have some actually | 14:53 |
| OliverW | nova_nova_conf_overrides: | 14:53 |
| OliverW | libvirt: | 14:53 |
| OliverW | iscsi_use_multipath: true | 14:53 |
| OliverW | nova_system_user_uid: 999 | 14:53 |
| OliverW | nova_system_group_gid: 999 | 14:53 |
| OliverW | these are the nova related ones | 14:53 |
| noonedeadpunk | that should be ok | 14:54 |
| noonedeadpunk | also can I suggest using http://paste.openstack.org/ for the futher pastes? | 14:54 |
| OliverW | sure, sorry | 14:55 |
| *** nurdie has quit IRC | 14:55 | |
| noonedeadpunk | can you also provide value of the physical_interface_mappings from /etc/neutron/plugins/ml2/linuxbridge_agent.ini on the compute host? | 14:55 |
| noonedeadpunk | and provider_networks structure from /etc/openstack_deploy/openstack_user_config.yml ? | 14:56 |
| OliverW | sure, here's the provider network structure: http://paste.openstack.org/show/800646/ | 14:57 |
| OliverW | and here's the linuxbridge_agent.ini: http://paste.openstack.org/show/800647/ | 14:59 |
| OliverW | ok, I just reproduced it and there are actually errors in the linuxbridge log: http://paste.openstack.org/show/800648/ | 15:05 |
| noonedeadpunk | oh, well, that's smth | 15:12 |
| noonedeadpunk | I see what's wring | 15:14 |
| noonedeadpunk | for `type: "flat"` you need to set real interface that you have on compute nodes as host_bind_override | 15:14 |
| *** nurdie has joined #openstack-ansible | 15:15 | |
| OliverW | I set up a veth pair because my compute nodes' hardware differs. That's why I used eth12 | 15:16 |
| *** nurdie has quit IRC | 15:19 | |
| noonedeadpunk | so your compute nodes does have eth12? | 15:20 |
| OliverW | yes it does | 15:21 |
| noonedeadpunk | oh | 15:21 |
| noonedeadpunk | jsut in case - does compute nodes have sudo installed? | 15:22 |
| OliverW | yep, just checked | 15:22 |
| noonedeadpunk | well eventually I'm wondering if the way you renamed interfaces does not allow to modify it and add it to the bridge... | 15:23 |
| OliverW | You mean with the veth pair? I think this was suggested somewhere in the docs for different physical interface names on the compute nodes | 15:24 |
| noonedeadpunk | yeah | 15:25 |
| noonedeadpunk | as eventually how I'd renamed interfaces would be adding to /etc/default/grub `GRUB_CMDLINE_LINUX=net.ifnames=0 biosdevname=0` and run update-grub | 15:26 |
| noonedeadpunk | that will result in regular eth0 eth1 etc interfaces after reboot | 15:26 |
| noonedeadpunk | I think we use veth pair only on controllers for lxc | 15:27 |
| OliverW | so the way to deal with different compute host hardware would be to create custom groups to segment them instead of the veth? | 15:29 |
| noonedeadpunk | or that | 15:30 |
| noonedeadpunk | I believe you can redefine interface for the flat network per group or per host with group/host vars | 15:31 |
| OliverW | oh, I didn't understand that about /etc/default/grub, now I got it! This is probably keeps the openstack_user_config.yml cleaner | 15:32 |
| OliverW | because this way I can stick with the predefined groups | 15:32 |
| OliverW | anyhow, thanks for your help! I'll check these options out | 15:33 |
| *** nurdie has joined #openstack-ansible | 15:51 | |
| admin0 | spatel, around ? | 15:57 |
| spatel | yup | 15:58 |
| admin0 | https://gist.github.com/a1git/c0fcad7a87a1366753c68d5bce09ad13 -- octavia :) | 15:58 |
| admin0 | using your blog, but my setup is a bit diff | 15:58 |
| *** mgariepy has quit IRC | 15:58 | |
| noonedeadpunk | I think that br-lbaas should not be configured on computes | 16:00 |
| noonedeadpunk | that is smth that neutron should bring up | 16:00 |
| spatel | not very good with netplan but what is this interfaces: [ bond0.1022 ] ? | 16:00 |
| spatel | You shouldn't configure any br-lbaas vlan interface on your controller and compute | 16:01 |
| noonedeadpunk | and why octavia_provider_segmentation_id is 101 and bond0.1022 ? | 16:02 |
| spatel | I have just created br-lbaas and attached with br-vlan using my script with veth pair | 16:02 |
| spatel | let me show you | 16:02 |
| noonedeadpunk | well or that | 16:02 |
| *** nurdie has quit IRC | 16:03 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Update octavia messaging options https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/765166 | 16:03 |
| spatel | admin0: http://paste.openstack.org/show/800651/ | 16:05 |
| spatel | I didn't create bond0.62 interface for VLAN | 16:05 |
| spatel | just attached br-lbaas bridge with br-vlan to carry my vlan62 traffic | 16:06 |
| admin0 | my neutron does not have the lbaas mapping .. physical_interface_mappings = vlan:br-vlan | 16:07 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Set reasonable default for octavia_ceilometer_enabled https://review.opendev.org/c/openstack/openstack-ansible/+/765167 | 16:07 |
| admin0 | is it possible to skip the br-lbaas and just create a network on vlan like any provider network ? | 16:08 |
| *** luksky has quit IRC | 16:08 | |
| spatel | you need br-lbaas bridge i believe | 16:09 |
| admin0 | can yuo share your physical_interface_mappings | 16:09 |
| admin0 | i think mine is missing and that is the issue | 16:09 |
| spatel | goal is to make br-lbaas subnet ip routable to compute nodes | 16:09 |
| admin0 | my compute nodes are also network nodes, so they already have br-lbaas | 16:10 |
| spatel | admin0: http://paste.openstack.org/show/800653/ | 16:11 |
| *** redrobot has quit IRC | 16:11 | |
| *** nurdie has joined #openstack-ansible | 16:11 | |
| admin0 | that i have .. i meant the /etc/neutron/plugins/ml2/linuxbridge_agent.ini mapping | 16:12 |
| spatel | I don't have any br-lbaas bridge on compute nodes, my amphora use br-vlan to attach vlan62 to route traffic to controller | 16:12 |
| admin0 | my thought was adding any --network block in the config is converted into mapping by neutron . its not the case ? | 16:12 |
| spatel | admin0: http://paste.openstack.org/show/800655/ | 16:13 |
| admin0 | so you run /usr/local/bin/setup-lbaas-veth.sh manually at boot ? | 16:14 |
| spatel | using /etc/rc.local to run at startup | 16:14 |
| admin0 | ok | 16:14 |
| admin0 | did you created the network or had ocatavia create it ? | 16:15 |
| admin0 | the script | 16:15 |
| admin0 | so you added it like a normal tagged network .. and then pluged in to the vlan using those veth | 16:15 |
| spatel | You have two method create br-lbaas bridge on each controller/compute etc.. to make it routable or use my method with use br-vlan to route traffic so you don't need to create br-lbaas on each nodes | 16:15 |
| spatel | admin0: yes that is correct | 16:16 |
| admin0 | my issue is br-lbaas is already there, but neutron is not seeing it as *any* interface | 16:16 |
| admin0 | so its preventing me to add any range on it | 16:16 |
| admin0 | my thought was when added as -network block, neutron would map it to a physical bridge/interface | 16:17 |
| spatel | if you see brctl output you will see v-br-lbaas.62 attach to br-lbaas and v-br-vlan attached to br-vlan | 16:17 |
| spatel | http://paste.openstack.org/show/800651/ | 16:17 |
| *** cshen has joined #openstack-ansible | 16:17 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_ceilometer master: Add octavia support to ceilometer https://review.opendev.org/c/openstack/openstack-ansible-os_ceilometer/+/765169 | 16:21 |
| spatel | I don't have this line in my config so not sure what it does - https://gist.github.com/a1git/c0fcad7a87a1366753c68d5bce09ad13#file-gistfile1-txt-L20 | 16:21 |
| spatel | admin0: this is what i have http://paste.openstack.org/show/800656/ | 16:23 |
| spatel | you are getting that error during creating this network may be - http://paste.openstack.org/show/800657/ | 16:25 |
| *** OliverW has quit IRC | 16:27 | |
| *** macz_ has joined #openstack-ansible | 16:33 | |
| admin0 | i understand what you did .. in my case, since br-lbaas already exists, i am going to add it as flat and see if that works | 16:34 |
| *** cshen has quit IRC | 16:37 | |
| *** mgariepy has joined #openstack-ansible | 16:37 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Decrease amphora image RAM for CI https://review.opendev.org/c/openstack/openstack-ansible/+/765134 | 16:38 |
| noonedeadpunk | yeah it feels ara really breaks deployments in CI now like it was a bit before with https://github.com/ansible-community/ara/commit/bba0f8143cdf3f0875531ba0048d65a71ea41df5 | 16:39 |
| noonedeadpunk | dmsimard: ^ | 16:39 |
| dmsimard | hi | 16:40 |
| dmsimard | nothing changed though | 16:40 |
| dmsimard | 1.5.3 has been out for a few weeks and I haven't had the time to work on ara a whole lot | 16:40 |
| dmsimard | did something change all of a sudden ? | 16:40 |
| noonedeadpunk | timeout on absolutely plain ground http://paste.openstack.org/show/800658/ | 16:40 |
| noonedeadpunk | I guess that set of dependencies might change | 16:41 |
| noonedeadpunk | (ie constraints) | 16:41 |
| dmsimard | yeah jrosser mentioned a sqlite database lock issue yesterday | 16:41 |
| noonedeadpunk | but have no idea honestly what goes wrong... maybe some package update or ci image... | 16:42 |
| dmsimard | I must say I have not personally seen an uptick of errors on my end | 16:43 |
| dmsimard | do you know when this might have started ? | 16:43 |
| noonedeadpunk | I saw timeouts randomly during previous week but never checked them out.... | 16:44 |
| dmsimard | noonedeadpunk: have a link to where ara is installed with it's deps ? can find out if there's anything out of the ordinary from the versions installed | 16:44 |
| openstackgerrit | James Denton proposed openstack/openstack-ansible master: Make container_bridge optional for provider networks https://review.opendev.org/c/openstack/openstack-ansible/+/765174 | 16:47 |
| dmsimard | noonedeadpunk: also for the record, the database lock issue is different from what we had with the multithreaded problem that the commit you linked intended to fix | 16:47 |
| dmsimard | the multithread issue was nasty, it'd seemingly lock up the process without further output or tracebacks | 16:48 |
| noonedeadpunk | uh, can't find :( | 16:54 |
| noonedeadpunk | dmsimard: copied from zuul console http://paste.openstack.org/show/800659/ | 16:57 |
| *** Guest18921 has joined #openstack-ansible | 16:58 | |
| *** gyee has joined #openstack-ansible | 16:58 | |
| noonedeadpunk | (the same one that I got previous paste with stack trace) | 16:58 |
| *** Guest18921 is now known as redrobot | 17:01 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Update octavia messaging options https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/765166 | 17:01 |
| dmsimard | noonedeadpunk: nothing jumps at me as being terribly wrong -- I see that django is 2.2.15 (released in august) although latest release is 2.2.17 (released in november) but there's nothing in django relnotes that mentions sqlite issues | 17:02 |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Define condition for the first play host one time https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/754411 | 17:04 |
| *** rpittau is now known as rpittau|afk | 17:04 | |
| dmsimard | it would be helpful to find out if there are similarities between the jobs where we see the issue (same repo, same distro, same branch, same job, etc.) | 17:07 |
| dmsimard | I tried to search for the sqlite locked error in logstash.o.o but it doesn't turn up anything | 17:07 |
| openstackgerrit | Merged openstack/openstack-ansible-os_senlin master: Fix senlin health manager group name https://review.opendev.org/c/openstack/openstack-ansible-os_senlin/+/764989 | 17:09 |
| admin0 | installation went on fine .. network is created .. but loadbalancer create failed | 17:16 |
| admin0 | do we have to create/use own amphora images and not rely on the default provided ? | 17:16 |
| admin0 | and i could not find logs | 17:16 |
| *** luksky has joined #openstack-ansible | 17:18 | |
| guilhermesp | admin0: i used to create following https://docs.openstack.org/octavia/latest/admin/amphora-image-build.html#building-images-for-alternate-branches | 17:18 |
| ThiagoCMC | Quick Ceph question! lol - According to the Ceph pgcalc, my pools must have: images=16 PGs, vms=32 GPs, backups=64 PGs, volumes=128PGs. Where exactly is the best place under /etc/openstack_deploy to put those values? OSA is setting 8 PG (pg_num) for all pools and, if I manually change it at the ceph-mon, a second run of OSA will un-do my changes. | 17:28 |
| *** nurdie has quit IRC | 17:30 | |
| *** cshen has joined #openstack-ansible | 17:41 | |
| *** cshen has quit IRC | 17:45 | |
| *** cshen has joined #openstack-ansible | 17:47 | |
| *** sshnaidm|ruck is now known as sshnaidm|afk | 17:47 | |
| *** nurdie has joined #openstack-ansible | 18:17 | |
| *** andrewbonney has quit IRC | 18:19 | |
| mgariepy | ThiagoCMC, https://docs.ceph.com/en/latest/rados/operations/placement-groups/ | 18:36 |
| mgariepy | pg_autoscale ? | 18:36 |
| *** pto has joined #openstack-ansible | 18:37 | |
| ThiagoCMC | mgariepy, I don't want pg_autoscale... I'm looking for a proper way to override the defaults, likely that they're coming from: `/etc/ansible/roles/ceph-ansible/roles/ceph-defaults/defaults/main.yml` or somewhere close. According to that file, they all use the same "osd_pool_default_pg_num"! But I don't want that (not recommended by pgcalc). | 18:39 |
| ThiagoCMC | Mayne I must duplicate part (the pools declarations?) of the file: "/etc/ansible/roles/ceph-ansible/roles/ceph-defaults/defaults/main.yml" within the file: "/etc/openstack_deploy/user_variables-ceph.yml" - but I'm not sure if this is the best approach. | 18:42 |
| *** pto has quit IRC | 18:43 | |
| *** miloa has quit IRC | 18:43 | |
| *** pto has joined #openstack-ansible | 18:43 | |
| *** cshen has quit IRC | 18:44 | |
| *** pto has quit IRC | 18:47 | |
| ThiagoCMC | Another question... :-P | 19:04 |
| ThiagoCMC | When "setup-everything.yml" finishes without error, and the command `openstack endpoint list` works within the Utility container BUT, Horizon login prints: "Something went wrong! An unexpected error has occurred."... What to do? The "os-horizon-install.yml" works... | 19:05 |
| ThiagoCMC | Fresh install OSA stable/ussuri branch | 19:05 |
| openstackgerrit | Merged openstack/openstack-ansible master: Update variables for default Zun deployments and AIO https://review.opendev.org/c/openstack/openstack-ansible/+/763562 | 19:11 |
| *** cshen has joined #openstack-ansible | 19:22 | |
| *** cozyurt has joined #openstack-ansible | 19:31 | |
| *** cozyurt has quit IRC | 19:39 | |
| ThiagoCMC | BTW, destroying/recreating all Horizon containers didn't fixed the login, I'm still seeing: "Something went wrong! An unexpected error has occurred."... So frustrating... I'll re-deploy, again. | 19:40 |
| *** vesper11 has joined #openstack-ansible | 20:15 | |
| *** vesper has joined #openstack-ansible | 20:19 | |
| *** vesper11 has quit IRC | 20:19 | |
| *** gshippey has joined #openstack-ansible | 20:24 | |
| admin0 | ThiagoCMC, i have seen it before .. a full run again fixed it | 20:40 |
| admin0 | like setup hosts, setup infra, setup os again | 20:40 |
| ThiagoCMC | Damn... lol | 20:40 |
| ThiagoCMC | Thanks! But I'm re-deploying :-D | 20:40 |
| *** waverider has joined #openstack-ansible | 21:04 | |
| *** vesper has quit IRC | 21:06 | |
| *** gregwork has joined #openstack-ansible | 21:09 | |
| *** rfolco has quit IRC | 21:20 | |
| openstackgerrit | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Reduce number of processes on small systems https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/764647 | 21:22 |
| *** vesper11 has joined #openstack-ansible | 21:30 | |
| *** waverider has quit IRC | 21:36 | |
| *** waverider has joined #openstack-ansible | 21:37 | |
| *** pto has joined #openstack-ansible | 21:51 | |
| *** pto has quit IRC | 21:55 | |
| *** stduolc has quit IRC | 22:00 | |
| *** stduolc has joined #openstack-ansible | 22:00 | |
| *** cshen has quit IRC | 22:08 | |
| jamesdenton | noonedeadpunk have you seen any issues with the git_requirements module and git reset --force not working? | 22:16 |
| *** yann-kaelig has joined #openstack-ansible | 22:16 | |
| *** cshen has joined #openstack-ansible | 22:17 | |
| *** yann-kaelig has quit IRC | 22:21 | |
| *** jbadiapa has quit IRC | 22:39 | |
| *** gshippey has quit IRC | 23:14 | |
| *** macz_ has quit IRC | 23:16 | |
| spatel | admin0: i am using default amphora image provided by CentOS community | 23:30 |
| *** cshen has quit IRC | 23:31 | |
| *** nurdie has quit IRC | 23:34 | |
| *** spatel has quit IRC | 23:56 | |
| *** luksky has quit IRC | 23:57 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!
