| opendevreview | Merged openstack/kolla-ansible stable/2023.1: Set HAProxy server timeout for openstack exporter https://review.opendev.org/c/openstack/kolla-ansible/+/901288 | 00:02 |
|---|---|---|
| opendevreview | Merged openstack/kolla-ansible stable/zed: Set HAProxy server timeout for openstack exporter https://review.opendev.org/c/openstack/kolla-ansible/+/901289 | 00:02 |
| opendevreview | Merged openstack/kolla-ansible stable/zed: Improve grok pattern matching for apache logs https://review.opendev.org/c/openstack/kolla-ansible/+/900075 | 00:03 |
| opendevreview | Verification of a change to openstack/kolla-ansible stable/zed failed: Add command to force reset the state of RabbitMQ https://review.opendev.org/c/openstack/kolla-ansible/+/901070 | 00:35 |
| opendevreview | Merged openstack/kolla-ansible stable/2023.1: ovn: Fix wording in limit check https://review.opendev.org/c/openstack/kolla-ansible/+/900994 | 01:59 |
| opendevreview | Merged openstack/kolla-ansible stable/zed: Add command to force reset the state of RabbitMQ https://review.opendev.org/c/openstack/kolla-ansible/+/901070 | 02:16 |
| opendevreview | Merged openstack/kolla-ansible stable/zed: ovn: Fix wording in limit check https://review.opendev.org/c/openstack/kolla-ansible/+/900995 | 02:41 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: fluentd: Use labels for transition to v5 https://review.opendev.org/c/openstack/kolla-ansible/+/901509 | 06:58 |
| opendevreview | Merged openstack/kolla-ansible master: Deprecate Masakari https://review.opendev.org/c/openstack/kolla-ansible/+/901193 | 07:32 |
| sylvr | Hello again, I'm here for the same problems as yesterday (kayobe deployment of bifrost failing) | 08:37 |
| sylvr | Some detail about the error : playbook fail when trying to deploy the bifrost services (kayobe seed service deploy) TASK [bifrost : Bootstrap bifrost (this may take several minutes)]. There's a huge wall of error (several screens worth of text). | 09:07 |
| sylvr | is there a way I can reduce the amount of log just for this task ? (the logfile is 1.2MB) or any pointer on how to troubleshoot the issue ? | 09:07 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: fluentd: Use labels for transition to v5 https://review.opendev.org/c/openstack/kolla-ansible/+/901509 | 09:10 |
| mnasiadka | sylvr: I don't believe you can, that's a whole blob of Ansible playbook run inside of a container | 09:19 |
| mnasiadka | but you can run the same command that Kolla-Ansible uses to deploy bifrost | 09:20 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: fluentd: Use labels for transition to v5 https://review.opendev.org/c/openstack/kolla-ansible/+/901509 | 09:21 |
| sylvr | mnasiadka: hum... I'll try that, how can I troubleshoot the issue then? what could help me? | 09:21 |
| mnasiadka | https://github.com/openstack/kolla-ansible/blob/57a8ff033fbb142d7980d99f3dc461ccc4eb42ed/ansible/roles/bifrost/tasks/bootstrap.yml#L16 | 09:22 |
| mnasiadka | this is the command we run in the container to bootstrap bifrost | 09:22 |
| mnasiadka | once you run that on your own (outside of Kayobe and Kolla-Ansible) - you should get a better understanding of what is happening | 09:23 |
| mnasiadka | feel free to raise a bug with that output - or just share it on paste.openstack.org and send a link here | 09:23 |
| mnasiadka | we'll try to help | 09:23 |
| sylvr | thanks ! | 09:26 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: DNM: Build all images https://review.opendev.org/c/openstack/kolla/+/901531 | 09:59 |
| sylvr | so, running the task directly on the seed container resulted in this : | 10:07 |
| sylvr | https://pastebin.com/1VJeNQef | 10:07 |
| sylvr | I'm going to try to get inside the container and see why it failed (checking systemd logs) | 10:08 |
| sylvr | seems like the DHCP range isn't okay | 10:11 |
| sylvr | https://pastebin.com/zKZmDB0a | 10:12 |
| sylvr | this is what's inside my etc/kayobe/networks.yml file : https://pastebin.com/HTGDXeYx | 10:18 |
| mnasiadka | you need to have inspection allocation pool configured | 10:33 |
| mnasiadka | and it can't be the same as allocation pool | 10:34 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: fluentd: Use labels for transition to v5 https://review.opendev.org/c/openstack/kolla-ansible/+/901509 | 10:38 |
| opendevreview | Merged openstack/kolla-ansible master: Adding mariadb_port to wsrep sync status so alterative ports can be used https://review.opendev.org/c/openstack/kolla-ansible/+/886581 | 11:58 |
| sylvr | mnasiadka: for every network or only the oob network ? because I have configured inspection allocation pool but it's a subset of the normal allocation poo | 12:21 |
| mnasiadka | sylvr: only the network you use for provisioning | 13:33 |
| sylvr | I don't know why, but the dhcp range in dsnmasq conf file is generated incorrectly : "dhcp-range=,,,12h" | 13:51 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: Support python bootstrap through Apt proxy https://review.opendev.org/c/openstack/kayobe/+/901551 | 13:52 |
| kickjay | Hi Kollas, Have you ever tried to run deployment (and other managment tasks) via AWX or Ansible Tower (with custom execution environment)? | 13:53 |
| mnasiadka | sylvr: because we don't have a precheck that checks if that is defined, so it templates out empty | 13:59 |
| sylvr | Am I supposed to fill in the blanks ? | 14:01 |
| sylvr | or is it due to misconfiguration? | 14:01 |
| mnasiadka | due to misconfiguration | 14:13 |
| sylvr | this is weird, my networks are configured in networks.yml | 14:16 |
| *** ravlew is now known as Guest7716 | 14:21 | |
| mattcrees | sylvr: the dhcp range used in Bifrost is templated down from `kolla_bifrost_dhcp_pool_start/end` here: https://github.com/openstack/kayobe/blob/8da3481ca236ade0a0a801f18d82153d457a7948/ansible/kolla-bifrost.yml#L10 | 14:23 |
| mattcrees | So you need to have an allocation pool set for your provision_oc_net | 14:23 |
| sylvr | yes, so it should depend on what is inside my etc/koayobe/networks.yml right ? | 14:26 |
| mattcrees | Yes exactly. Looking at the networks.yml you sent earlier, is oob your provision net? So you've set `provision_oc_net_name: "oob"` | 14:31 |
| opendevreview | Pierre Riteau proposed openstack/kolla master: Fix documented OpenSearch version https://review.opendev.org/c/openstack/kolla/+/901554 | 14:40 |
| sylvr | yes it is ! | 14:41 |
| mattcrees | In that case, it should get templated into bifrost.yml during the kolla-bifrost role. I.e. when you run `kayobe seed service deploy` | 14:44 |
| sylvr | this is weird | 14:44 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: fluentd: Use labels for transition to v5 https://review.opendev.org/c/openstack/kolla-ansible/+/901509 | 14:44 |
| opendevreview | Pierre Riteau proposed openstack/kolla master: Fix documented OpenSearch version https://review.opendev.org/c/openstack/kolla/+/901554 | 14:48 |
| mattcrees | If you check under {{ kolla_node_custom_config_path }}/bifrost/bifrost.yml, are dhcp_pool_start and dhcp_pool_end set there? You can run `kayobe seed service deploy -t kolla-bifrost -kt none` to just generate the bifrost config locally if needed | 14:48 |
| mattcrees | kolla_node_custom_config_path is etc/kolla by default btw :) | 14:55 |
| mattcrees | *etc/kolla/config | 14:56 |
| sylvr | they're not set on the node | 14:57 |
| sylvr | I'm running all the playbook to see if this change anything | 14:57 |
| opendevreview | Merged openstack/kayobe stable/yoga: Fix setting kolla_admin_openrc_cacert https://review.opendev.org/c/openstack/kayobe/+/900586 | 15:00 |
| jovial | sylvr: What does `kayobe configuration dump --limit localhost --var-name provision_oc_net_name` give you? | 15:26 |
| sylvr | jovial: this { "localhost": "oob" } | 15:59 |
| opendevreview | Merged openstack/kolla-ansible master: Remove Glance and Neutron TLS proxies from fluentd input config https://review.opendev.org/c/openstack/kolla-ansible/+/890085 | 16:08 |
| sylvr | okay, somehow the config is overriden | 16:15 |
| sylvr | bc the networks.yml file is correct (or at least it contain the correct info) but then on the seed there isn't any information about the allocation pool | 16:19 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: fluentd: fix horizon format regex https://review.opendev.org/c/openstack/kolla-ansible/+/890128 | 16:27 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: fluentd: fix horizon format regex https://review.opendev.org/c/openstack/kolla-ansible/+/890128 | 16:28 |
| opendevreview | Dawud proposed openstack/kolla-ansible master: Enable HAProxy Prometheus metrics endpoint https://review.opendev.org/c/openstack/kolla-ansible/+/877118 | 16:39 |
| opendevreview | Dawud proposed openstack/kolla-ansible master: Enable HAProxy Prometheus metrics endpoint https://review.opendev.org/c/openstack/kolla-ansible/+/877118 | 16:40 |
| sylvr | good night, coming back tomorrow ! ;) | 16:40 |
| opendevreview | Merged openstack/kayobe master: provision-net: Remove transitional unshare code https://review.opendev.org/c/openstack/kayobe/+/899936 | 19:39 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: Use Inspector database to store inspection data https://review.opendev.org/c/openstack/kayobe/+/851834 | 20:39 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: Remove custom grafana_local_admin_user_name https://review.opendev.org/c/openstack/kayobe/+/877445 | 20:45 |
| opendevreview | Pierre Riteau proposed openstack/kolla-ansible master: Move [oslo_policy] back inside Jinja if block https://review.opendev.org/c/openstack/kolla-ansible/+/901629 | 21:39 |
| opendevreview | Pierre Riteau proposed openstack/kolla master: Update Ceph repo name for dnf config-manager command https://review.opendev.org/c/openstack/kolla/+/901634 | 22:45 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: Test Kolla fix https://review.opendev.org/c/openstack/kayobe/+/901635 | 22:47 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: fluentd: Use labels for transition to v5 https://review.opendev.org/c/openstack/kolla-ansible/+/901509 | 22:52 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: Enable HAProxy Prometheus metrics endpoint https://review.opendev.org/c/openstack/kolla-ansible/+/877118 | 22:54 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!