| *** mazzy5098812929580859404 is now known as mazzy50988129295808594 | 03:14 | |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: [release] Sync Kolla Ansible feature flags for Yoga https://review.opendev.org/c/openstack/kayobe/+/834458 | 05:30 |
|---|---|---|
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/xena: Sync Kolla Ansible feature flags for Xena backports https://review.opendev.org/c/openstack/kayobe/+/835933 | 05:33 |
| holtgrewe | Hello. I have a problem with container storage growing and growing. A "bad" one is elasticsearch. How do I clean this out? | 05:36 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/xena: Sync Kolla Ansible feature flags and inventory https://review.opendev.org/c/openstack/kayobe/+/835933 | 05:37 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla stable/wallaby: macros/pip: revert to old setuptools way https://review.opendev.org/c/openstack/kolla/+/835872 | 07:53 |
| opendevreview | Mark Goddard proposed openstack/kayobe master: CI: separate image builds into a non-voting job https://review.opendev.org/c/openstack/kayobe/+/835610 | 08:52 |
| jingvar | morning | 08:56 |
| mgoddard | holtgrewe: enable elasticsearch curator | 08:59 |
| opendevreview | Merged openstack/kayobe stable/xena: Fix custom config idempotence https://review.opendev.org/c/openstack/kayobe/+/835802 | 09:09 |
| opendevreview | Merged openstack/kayobe stable/wallaby: Fix custom config idempotence https://review.opendev.org/c/openstack/kayobe/+/835803 | 09:09 |
| opendevreview | Merged openstack/kayobe stable/victoria: Fix custom config idempotence https://review.opendev.org/c/openstack/kayobe/+/835804 | 09:09 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/xena: CI: fix TLS job by freeing up memory https://review.opendev.org/c/openstack/kayobe/+/835951 | 09:16 |
| mirek186 | are there any extra steps required to get Monasca working from Horizon, when click on overview tab got HTTP 500 error with "No policy rules for service 'monitoring' in /etc/openstack-dashboard/monitoring_policy.yaml" and python exception AttributeError: 'Settings' object has no attribute 'DASHBOARDS' | 09:20 |
| mirek186 | when I check Horizon container I can see there is no monitoring_policy.yaml file, but not sure if there should be one or not. | 09:21 |
| opendevreview | Mark Goddard proposed openstack/kolla master: [release] Use Yoga sources by default https://review.opendev.org/c/openstack/kolla/+/835846 | 09:26 |
| opendevreview | Mark Goddard proposed openstack/kolla master: [release] Use RDO's Yoga Delorean packages https://review.opendev.org/c/openstack/kolla/+/835847 | 09:26 |
| opendevreview | Mark Goddard proposed openstack/kayobe master: libvirt: Enable host daemon by default https://review.opendev.org/c/openstack/kayobe/+/835034 | 09:35 |
| mirek186 | there is a change in monasca-ui introduced 3rd feb changing json to yaml. That's why is failing to copy correct policy file in horizon: https://github.com/openstack/monasca-ui/commit/d062507ef5794fbc8720eeee7d0f9804e23c3136 | 10:16 |
| yuval | seems like there were a lot of things going on lately | 10:24 |
| yuval | the jinja version killed my ci - jinja==3.1 | 10:24 |
| opendevreview | Pierre Riteau proposed openstack/kolla master: Fix disabling of curlrc in healthcheck_curl https://review.opendev.org/c/openstack/kolla/+/835957 | 10:24 |
| yuval | if anyway have problems with it you should install jinja==3.0.3 | 10:24 |
| yuval | anyone* | 10:25 |
| amoralej | hi hrw, you use rdo packages for aarch64 somewhere? i think we have a problem in the packages for that arch | 10:25 |
| yuval | if kolla ansible angery about something to do with "filter" | 10:25 |
| hrw | amoralej: we have an option of using them but do not use on CI. | 11:11 |
| hrw | amoralej: I can run a build to check do images build but lack systems to deploy | 11:11 |
| amoralej | hrw, i was just trying to test a package installation, what's the best way to run an aarch64 test system? | 11:12 |
| amoralej | i'm trying to emulate with qemu | 11:12 |
| amoralej | but fails to boot | 11:12 |
| hrw | amoralej: give me half an hour - started a build of all images to check do they build | 11:13 |
| amoralej | ack, fyi, my guess is that it will fail to build because of openstack-nova-compute | 11:13 |
| hrw | ok, will limit to nova then | 11:14 |
| hrw | and if it work then build rest | 11:14 |
| amoralej | hrw, https://github.com/rdo-packages/nova-distgit/blob/rpm-master/openstack-nova.spec#L200 | 11:14 |
| hrw | amoralej: Yoga is the last release where we use binary packages for openstack components | 11:14 |
| amoralej | it seems that doesn't exist for aarch64 | 11:15 |
| hrw | lovely | 11:15 |
| hrw | I wonder does it work outside of x86-64 at all | 11:15 |
| amoralej | apparently nova can work without it but i'm not sure how to handle that in packaging | 11:15 |
| amoralej | as the package is noarch | 11:16 |
| mirek186 | I've got Monasca UI working, I tought I could propose a fix however with my limited knowlage for policies and how Horizon is loading everything I'll open a bug just so someone can have a look first | 11:19 |
| mirek186 | so there might be two issues here, one local_settings.py is not copied over from monasca-ui config dir | 11:20 |
| mirek186 | two the policy file is missing but when I fix it the role the file implement is not in keystone and it won't show the tab at all, when I comment out the roles it's working | 11:20 |
| mirek186 | sorry the UI is working not the role :) | 11:21 |
| hrw | ERROR:kolla.common.utils:nova-compute Failed with status: error | 11:27 |
| hrw | ERROR:kolla.common.utils:nova-compute-ironic Failed with status: error | 11:27 |
| hrw | amoralej: +1 | 11:27 |
| hrw | INFO:kolla.common.utils.nova-compute: - nothing provides /usr/lib64/qemu-kvm/hw-display-virtio-vga.so needed by openstack-nova-compute-1:25.1.0-0.20220330074311.b0851b0.el8.noarch | 11:27 |
| amoralej | yeah :) | 11:27 |
| amoralej | as expected | 11:27 |
| amoralej | damn | 11:28 |
| hrw | rdo-- ;P | 11:28 |
| amoralej | :( | 11:28 |
| hrw | or-whoever-maintain-the-package-- | 11:28 |
| amoralej | there are logs somewhere that i can link? | 11:28 |
| hrw | amoralej: https://paste.centos.org/view/c6619ff7 | 11:29 |
| amoralej | thanks, i'm filling a bz | 11:30 |
| hrw | thx | 11:31 |
| amoralej | it's not an easy one | 11:31 |
| hrw | amoralej: cc: mjuszkie@redhat.com | 11:31 |
| amoralej | ack, i will | 11:31 |
| mirek186 | I've open a bug for monasca, https://bugs.launchpad.net/kolla-ansible/+bug/1967278 happy to provide the patch as long as someone can confirm that my assumptions are correct | 11:33 |
| hrw | if a fix worked for you then please send patch | 11:36 |
| mirek186 | I can access the overview tab now, but the part I'm not clear about is current version tries to copy monitoring_policy.json, been renamed to monitoring_policy.yaml but in that file are roles which if not commented out not allow admin user to even see the monitoring tab | 11:39 |
| mirek186 | don't know enough about Horizon and Keystone policies to decide what to do with the file. Empty file does the trick to at least view it in Horizon | 11:40 |
| hrw | neither do I | 11:42 |
| hrw | mgoddard, mnasiadka, frickler: ^^? | 11:42 |
| mnasiadka | I'm not even a Monasca newbie, and I'm not planning to be :) | 11:43 |
| mnasiadka | And I don't think we ever used monasca-ui | 11:44 |
| mirek186 | the roles are here, when I look at roles in Horizon non of them are defined https://github.com/openstack/monasca-ui/blob/master/monitoring/conf/monitoring_policy.yaml | 11:44 |
| mirek186 | the closet one defined is monasca-read-only-user , so I wonder whether that role should be called monasca_user_role | 11:46 |
| hrw | mnasiadka: you know more about deployments than I do | 11:51 |
| mirek186 | there is another one when you try to create Alarm Definition, AttributeError: 'ExpressionWidget' object has no attribute 'metrics' | 11:53 |
| *** amoralej is now known as amoralej|lunch | 11:54 | |
| mnasiadka | As said, I don't know if monasca-ui has ever worked in Kolla-Ansible - mirek186 you're better of discussing that with Monasca team and proposing fixes to kolla-ansible | 11:54 |
| mnasiadka | but I don't know how much of a pulse Monasca has today. | 11:54 |
| mirek186 | mnasiadka: will do, a bit offtopic is Monasca here to stay, I just wonder taking into account it's java, kawka and quite a bit of infra whether it's worth investing my time | 11:58 |
| mnasiadka | mirek186: probably it's better to invest time in Prometheus - because it's more of a common use case | 12:00 |
| mirek186 | mnasiadka: thanks mate, we alrady have Prometheus and Grafan accross diffrent projects plus monitoring enabled in Kolla. Is there any extra value when using Monasca and it's integration with OpenStack, or if it is just another monitoring tool then sticking to Prometheus dose make sense | 12:06 |
| mnasiadka | another monitoring tool with not a lot of activity :) | 12:06 |
| mnasiadka | but that's only my opinion | 12:06 |
| hrw | and we have a bunch of prometheus exporters | 12:07 |
| mirek186 | the only reason I was willing to spend time on it, is part of OpenStack so I tought it might have a bit better intergration and wider use case but it looks like it's not worth spending much time, especially that the whole setup looks complicated and even basic deployment is problematic | 12:08 |
| hrw | I would not be surprised seeing more people trying it and then reverting to prometheus just because they know it already | 12:09 |
| mirek186 | I do get why would you use Kafa and everything however it's adding the complexity and almost another role of "who's watching the watcher" thing, were promethues in it's simpliciy usually just works :) | 12:11 |
| mnasiadka | mirek186: it's been designed to work at huge scale, but I 99% of deployments are not that big so it matters :) | 12:32 |
| *** amoralej|lunch is now known as amoralej | 13:06 | |
| opendevreview | Mark Goddard proposed openstack/kolla master: [release] Use Yoga sources by default https://review.opendev.org/c/openstack/kolla/+/835846 | 13:11 |
| opendevreview | Mark Goddard proposed openstack/kolla master: [release] Use RDO's Yoga Delorean packages https://review.opendev.org/c/openstack/kolla/+/835847 | 13:11 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: Run testssl.sh against HAProxy on tls scenario https://review.opendev.org/c/openstack/kolla-ansible/+/823499 | 13:12 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: Run testssl.sh against HAProxy on tls scenario https://review.opendev.org/c/openstack/kolla-ansible/+/823499 | 13:29 |
| opendevreview | Merged openstack/kayobe stable/xena: CI: fix TLS job by freeing up memory https://review.opendev.org/c/openstack/kayobe/+/835951 | 13:29 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/xena: Sync Kolla Ansible feature flags and inventory https://review.opendev.org/c/openstack/kayobe/+/835933 | 13:44 |
| frickler | mnasiadka: what would be the advantage of registering for the PTG? other then giving my private data to 3rd parties? | 13:44 |
| mnasiadka | frickler: I don't know, I'm just conveying the message from foundation that is sent to PTLs - probably they just harvest your data and subscribe you to some devilish plans ;-) (or just use this data to have an idea around number of people attending PTG) | 13:45 |
| frickler | oh, so the foundation pushed you to write that, interesting | 13:46 |
| amoralej | hrw, i hope https://review.rdoproject.org/r/c/openstack/nova-distgit/+/41230 will fix it but i need nova team to check | 13:56 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: haproxy: support single external frontend https://review.opendev.org/c/openstack/kolla-ansible/+/823395 | 14:19 |
| ebbex | Is there any documentation on how to make the kayobe/seed provide dhcp? | 14:38 |
| ebbex | I've got bifrost running in a container on it, but I might be missing dhcp-features? | 14:40 |
| opendevreview | Mark Goddard proposed openstack/kayobe master: Rework network interface to host mapping https://review.opendev.org/c/openstack/kayobe/+/825832 | 14:54 |
| jingvar | bifrost is ironic = dnsmasq | 14:54 |
| jingvar | pxe boot = dhcp+tftp+http | 14:56 |
| jingvar | (bifrost-deploy)[root@kayobe bifrost-11.2.1.dev6]# grep listen-address /etc/dnsmasq.conf | 14:57 |
| ebbex | aha, listen-address=192.168.3.8, which seems all good. but `ps aux` shows no dnsmasq process running in the container. | 14:59 |
| ironfoot | can this run in a single container? I'd say "no" | 15:01 |
| ebbex | ironfoot: ? | 15:04 |
| jingvar | container host5network | 15:04 |
| ironfoot | a container normally would run only one process, and not multiple services, right? | 15:06 |
| jingvar | no | 15:06 |
| jingvar | it is a way | 15:06 |
| ebbex | ironfoot: it can run plenty. the bifrost container has nginx mariadb, varous ironic services and xinetd. | 15:06 |
| jingvar | what happens when you run docker exec -ti som_container bash ? | 15:07 |
| ironfoot | right, ok | 15:08 |
| ebbex | jingvar: I don't have a host5network container on my kayobe/seed. | 15:08 |
| ebbex | Only got `kolla/centos-source-bifrost-deploy:wallaby` | 15:09 |
| jingvar | hostnetwork is a method to run container (host namespace) | 15:09 |
| ebbex | jingvar: I don't follow. is there a setting i forgot in my kayobe-config? Will this affect wether or not dnsmasq gets to run? | 15:12 |
| jingvar | I try to say, that if dnsmasq is running and properly configured it should work | 15:13 |
| jingvar | did you check that seed node has boottp packets on a interface | 15:14 |
| ebbex | well, it doesn't look like it's running inside the bifrost_deploy container. | 15:14 |
| jingvar | 40 ? Ss 1:19 /usr/sbin/dnsmasq -k | 15:15 |
| jingvar | docker exec -ti bifrost_deploy ps -a | 15:15 |
| jingvar | docker exec -ti bifrost_deploy ps -ax | 15:16 |
| ebbex | https://paste.opendev.org/show/813585/ | 15:16 |
| jingvar | have you finished bifrost startup | 15:16 |
| jingvar | try to run /usr/sbin/dnsmasq -k and watch logs | 15:19 |
| ebbex | I'm not sure? I'm following this guide https://docs.openstack.org/kayobe/wallaby/deployment.html and I've gotten to `kayobe seed deployment image build` | 15:19 |
| jingvar | :) | 15:20 |
| ebbex | I'm looking at the `Overcloud discovery` part now, but I might be missing a step or two, since there's no response to dhcp requests from my pxe booting servers. | 15:21 |
| jingvar | https://github.com/stackhpc/a-universe-from-nothing | 15:21 |
| ebbex | s/gotten to/gotten past/ | 15:21 |
| jingvar | It doesnt work from the box :) | 15:21 |
| opendevreview | Merged openstack/kolla-ansible master: rabbitmq: follow up for classic queue mirror removal https://review.opendev.org/c/openstack/kolla-ansible/+/834342 | 15:21 |
| jingvar | https://github.com/jingvar/a-universe-from-nothing/tree/test | 15:22 |
| opendevreview | Merged openstack/kolla-ansible master: Adds FluentD configuration for OpenvSwitch log matching https://review.opendev.org/c/openstack/kolla-ansible/+/834555 | 15:23 |
| opendevreview | Merged openstack/kolla master: [release] Tidy up release notes for Yoga https://review.opendev.org/c/openstack/kolla/+/835840 | 15:23 |
| jingvar | deployment imag build - I don't use | 15:25 |
| ebbex | jingvar: thanks for the link, I'm looking into the various commits. Do you have a previous background in openstack-ansible perhaps? | 15:39 |
| jingvar | no | 15:39 |
| jingvar | I hate ansible | 15:40 |
| ebbex | just the naming convention for your nets; mgmt, storage, tunnel | 15:43 |
| hrw | amoralej: on Debian this driver is enabled: https://packages.debian.org/bullseye-backports/arm64/qemu-system-common/filelist | 15:43 |
| opendevreview | Merged openstack/kayobe master: [release] Sync Kolla Ansible feature flags for Yoga https://review.opendev.org/c/openstack/kayobe/+/834458 | 15:46 |
| amoralej | hrw, from virt team in rh ""virtio-gpu is same as virtio-vga, but without the vga bits and so works on all arches, but requires guest support explicitly"" | 15:46 |
| amoralej | dunno what that means | 15:46 |
| hrw | amoralej: vga is x86 legacy anyway | 15:47 |
| hrw | amoralej: let me check which driver nova uses | 15:47 |
| hrw | x86/x86-64 uses virtio, ppc* uses vga, aarch64 uses virtio. and virtio-gpu can fallback to vga | 15:52 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla stable/wallaby: Always use the distro-provided libvirt-python https://review.opendev.org/c/openstack/kolla/+/835973 | 16:03 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla stable/wallaby: Always use the distro-provided libvirt-python https://review.opendev.org/c/openstack/kolla/+/835973 | 16:04 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla stable/wallaby: Always use the distro-provided libvirt-python https://review.opendev.org/c/openstack/kolla/+/835973 | 16:05 |
| opendevreview | Mark Goddard proposed openstack/kayobe master: Rework network interface to host mapping https://review.opendev.org/c/openstack/kayobe/+/825832 | 16:06 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla stable/wallaby: Always use the distro-provided libvirt-python https://review.opendev.org/c/openstack/kolla/+/835973 | 16:07 |
| opendevreview | Mark Goddard proposed openstack/kayobe master: [release] Sync Kolla Ansible inventory for Yoga https://review.opendev.org/c/openstack/kayobe/+/836038 | 16:17 |
| opendevreview | Mark Goddard proposed openstack/kayobe master: [release] Use OpenStack Yoga release https://review.opendev.org/c/openstack/kayobe/+/836040 | 16:21 |
| opendevreview | Mark Goddard proposed openstack/kayobe-config master: [release] Synchronise with kayobe changes for Yoga https://review.opendev.org/c/openstack/kayobe-config/+/836041 | 16:27 |
| ebbex | I was missing the inspection_allocation_pool ranges for dnsmasq to start :/ | 16:29 |
| ebbex | I didn't quite understand what they were for. But anyways now it's working! :) | 16:30 |
| ebbex | Excited to see how the discovery process will work | 16:30 |
| opendevreview | Marcin Juszkiewicz proposed openstack/kolla stable/wallaby: Always use the distro-provided libvirt-python https://review.opendev.org/c/openstack/kolla/+/835973 | 16:31 |
| opendevreview | Mark Goddard proposed openstack/kayobe-config-dev master: [release] Sync with kayobe changes for Yoga https://review.opendev.org/c/openstack/kayobe-config-dev/+/836042 | 16:32 |
| *** amoralej is now known as amoralej|off | 17:15 | |
| opendevreview | Merged openstack/kolla master: [release] Use Yoga sources by default https://review.opendev.org/c/openstack/kolla/+/835846 | 17:26 |
| opendevreview | Merged openstack/kolla master: [release] Use RDO's Yoga Delorean packages https://review.opendev.org/c/openstack/kolla/+/835847 | 18:00 |
| opendevreview | Merged openstack/kolla stable/xena: macros/pip: revert to old setuptools way https://review.opendev.org/c/openstack/kolla/+/835856 | 19:56 |
| admiyo | How does one go about debugging a failed deploy, specifically with OVS and neutron? The Containers are just restarting. As Far as I can tell, the actual OVS process has died, but left behind a pidfile and there is a constant stream of errors after that checking the pidfile and seeing there should be a process and there isn't. No real hard info on what caused the crash though. | 20:40 |
| admiyo | Ansible shows "Error response from daemon: Container 3b8501208e7cc64b579eb4ae17d2eea009c5780a96fa94c13f799039305ac7a5 is restarting, wait until the container is running" | 20:41 |
| admiyo | and the log from the container shows the pidfile error | 20:41 |
| admiyo | 2022-03-31T20:41:43.745Z|00002|daemon_unix|WARN|/var/run/openvswitch/ovsdb-server.pid: stale pidfile for pid 1735 | 20:42 |
| admiyo | being deleted by pid 0 | 20:42 |
| admiyo | 2022-03-31T20:41:43.745Z|00003|daemon_unix|EMER|/var/run/openvswitch/ovsdb-server.pid: pidfile check failed (No such process), aborting | 20:42 |
| *** edebeste8 is now known as edebeste | 22:17 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!