| dcapone2004 | In the process of catching up on upgrading some openstack environments and I looked to go from ussuri to victoria....everything went well except of course the nova_compute...I am getting a failed to connect to libvirt: authentication failure | 03:58 |
|---|---|---|
| dcapone2004 | the nova_libvirt container is up and healthy...I ensured that the new libvirt_sasl password was added to passwords.yml by following procedure with kolla-genpwd and kolla-mergepw... I have also looked in the /etc/kolla/config/nova-compute/auth.conf generated file and I can confirm that the password matches the password from passwords.yml | 04:07 |
| opendevreview | Tim Beermann proposed openstack/kolla-ansible master: Removed sanity checks https://review.opendev.org/c/openstack/kolla-ansible/+/839716 | 04:41 |
| frickler | dcapone2004: make sure your hostname matches what is recorded in DNS or /etc/hosts. if there is a mismatch /long vs. short) this issue may happen | 05:48 |
| *** amoralej|off is now known as amoralej | 06:19 | |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: cephadm: Add osds/pools/users in one run https://review.opendev.org/c/openstack/kolla-ansible/+/836838 | 06:54 |
| opendevreview | Tim Beermann proposed openstack/kolla-ansible master: Removed sanity checks https://review.opendev.org/c/openstack/kolla-ansible/+/839716 | 06:54 |
| mnasiadka | dcapone2004: are you sure you have fresh nova-compute and nova-libvirt container images? If they are from before that change, there are some packages missing in the images. | 07:41 |
| oklhost_ | I had the opportunity to write an article on my company's tech blog and do some promotion for OpenStack & Kolla/Kolla-Ansible. :) https://www.otto.de/jobs/technology/techblog/artikel/the-open-source-private-cloud-in-the-otto-group.php | 07:54 |
| frickler | moin oklhost_, nice article indeed, thanks for sharing | 08:07 |
| oklhost_ | thanks :) | 08:08 |
| frickler | oklhost_: what's this "own code we have written", just things you committed upstream or do you have additional stuff that would be interesting? | 08:14 |
| oklhost_ | frickler: Well, thats the adjutant-playbook we're still using, a registration app for customers, because we need some special tags at projects for our internal chargeback as well as a still ongoing project to calculate costs, but that isn't in production right now. Then there is a lot of other smaller scripts to deal with those german company's processes around everything. ;) | 08:21 |
| oklhost_ | Also ci/cd pipelines are meant by "own code". Which is perhaps stacked a bit too high. ;) | 08:25 |
| frickler | oklhost_: ah, o.k., I was just wondering when you claim "we have open sourced everything" whether there's a list of repos related to that | 08:25 |
| oklhost_ | frickler: that was related to the adjutant-playbook. I've written it a bit more clear, but the company communication shortened the article a lot. | 08:27 |
| oklhost_ | brb | 08:28 |
| frickler | oklhost_: fair enough, I didn't want to complain, just find out if I'd missed something :) | 08:30 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: cephadm: Add osds/pools/users in one run https://review.opendev.org/c/openstack/kolla-ansible/+/836838 | 08:42 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: cephadm: Add osds/pools/users in one run https://review.opendev.org/c/openstack/kolla-ansible/+/836838 | 08:48 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: cephadm: Add osds/pools/users in one run https://review.opendev.org/c/openstack/kolla-ansible/+/836838 | 09:09 |
| oklhost_ | frickler: np, I know, I just want be as open as possible ;) | 09:15 |
| opendevreview | Merged openstack/kayobe stable/xena: Bump stackhpc.drac role https://review.opendev.org/c/openstack/kayobe/+/839652 | 09:17 |
| opendevreview | Merged openstack/kayobe stable/victoria: Cleanup old and deprecated Swift configuration https://review.opendev.org/c/openstack/kayobe/+/839039 | 09:17 |
| opendevreview | Merged openstack/kayobe master: Use rsync to update kayobe-config https://review.opendev.org/c/openstack/kayobe/+/839482 | 09:17 |
| opendevreview | Merged openstack/kayobe master: CI: Add some more irrelevant files to zuul jobs https://review.opendev.org/c/openstack/kayobe/+/837836 | 09:17 |
| opendevreview | Merged openstack/kayobe master: Fix Bifrost inspection through DHCP-relay https://review.opendev.org/c/openstack/kayobe/+/834204 | 09:17 |
| kevko | yoctozepto: hi, what about my answer to your comment :P ? | 09:25 |
| kevko | *reply | 09:25 |
| opendevreview | Maksim Malchuk proposed openstack/kayobe stable/yoga: Fix Bifrost inspection through DHCP-relay https://review.opendev.org/c/openstack/kayobe/+/839883 | 09:33 |
| opendevreview | Maksim Malchuk proposed openstack/kayobe stable/xena: Fix Bifrost inspection through DHCP-relay https://review.opendev.org/c/openstack/kayobe/+/839884 | 09:34 |
| opendevreview | Maksim Malchuk proposed openstack/kayobe stable/wallaby: Fix Bifrost inspection through DHCP-relay https://review.opendev.org/c/openstack/kayobe/+/839885 | 09:34 |
| opendevreview | Merged openstack/kolla-ansible stable/ussuri: nova: use any_errors_fatal for once-per-cell tasks https://review.opendev.org/c/openstack/kolla-ansible/+/839176 | 09:40 |
| opendevreview | Merged openstack/kolla-ansible stable/victoria: nova: use any_errors_fatal for once-per-cell tasks https://review.opendev.org/c/openstack/kolla-ansible/+/839175 | 09:40 |
| opendevreview | Merged openstack/kolla-ansible stable/wallaby: nova: use any_errors_fatal for once-per-cell tasks https://review.opendev.org/c/openstack/kolla-ansible/+/839174 | 09:41 |
| opendevreview | Merged openstack/kolla-ansible stable/xena: nova: use any_errors_fatal for once-per-cell tasks https://review.opendev.org/c/openstack/kolla-ansible/+/839173 | 09:44 |
| hrw | morning | 09:45 |
| kevko | morning \o/ | 09:51 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: Use openstack-ci-mirrors in ipa image build https://review.opendev.org/c/openstack/kayobe/+/821675 | 09:51 |
| opendevreview | Merged openstack/kolla-ansible stable/yoga: nova: use any_errors_fatal for once-per-cell tasks https://review.opendev.org/c/openstack/kolla-ansible/+/839172 | 09:56 |
| opendevreview | Merged openstack/kolla-ansible stable/yoga: nova: improve compute service registration failure handling https://review.opendev.org/c/openstack/kolla-ansible/+/839177 | 09:56 |
| opendevreview | Merged openstack/kolla-ansible stable/xena: nova: improve compute service registration failure handling https://review.opendev.org/c/openstack/kolla-ansible/+/839331 | 09:58 |
| opendevreview | Merged openstack/kolla-ansible stable/wallaby: nova: improve compute service registration failure handling https://review.opendev.org/c/openstack/kolla-ansible/+/839178 | 09:58 |
| opendevreview | Merged openstack/kolla-ansible stable/victoria: nova: improve compute service registration failure handling https://review.opendev.org/c/openstack/kolla-ansible/+/839179 | 09:58 |
| opendevreview | Merged openstack/kolla-ansible stable/ussuri: nova: improve compute service registration failure handling https://review.opendev.org/c/openstack/kolla-ansible/+/839180 | 09:58 |
| opendevreview | Merged openstack/kolla master: Use rsync to update kayobe-config https://review.opendev.org/c/openstack/kolla/+/839496 | 09:58 |
| opendevreview | Merged openstack/kayobe master: kolla_passwords: add no_log for password overrides https://review.opendev.org/c/openstack/kayobe/+/837553 | 10:00 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/yoga: kolla_passwords: add no_log for password overrides https://review.opendev.org/c/openstack/kayobe/+/839744 | 10:01 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/xena: kolla_passwords: add no_log for password overrides https://review.opendev.org/c/openstack/kayobe/+/839745 | 10:01 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/wallaby: kolla_passwords: add no_log for password overrides https://review.opendev.org/c/openstack/kayobe/+/839746 | 10:01 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/victoria: kolla_passwords: add no_log for password overrides https://review.opendev.org/c/openstack/kayobe/+/839747 | 10:02 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/ussuri: kolla_passwords: add no_log for password overrides https://review.opendev.org/c/openstack/kayobe/+/839748 | 10:02 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/train: kolla_passwords: add no_log for password overrides https://review.opendev.org/c/openstack/kayobe/+/839749 | 10:02 |
| opendevreview | Merged openstack/kolla master: Add multipath to cinder-volume https://review.opendev.org/c/openstack/kolla/+/839481 | 10:09 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: cephadm: Add osds/pools/users in one run https://review.opendev.org/c/openstack/kolla-ansible/+/836838 | 10:51 |
| mgoddard | fg | 11:08 |
| SvenKieske | is it okay to promote a free barcamp/meetup here, organized by my employer? | 11:48 |
| dcapone2004 | frickler: dont think dns is the issue ss all other tooling works | 12:00 |
| frickler | dcapone2004: its not about dns per se, the hostname is used as part of the username, let me dig the command to check this | 12:01 |
| dcapone2004 | mnasiadka: i ran a pull before upgrade so they should be the latest victoria...also the container names all updated from ussuri to victoria | 12:01 |
| opendevreview | Merged openstack/kayobe stable/victoria: Bump stackhpc.drac role https://review.opendev.org/c/openstack/kayobe/+/839700 | 12:01 |
| dcapone2004 | i see....the hostnames are all in /etc/hosts as well | 12:02 |
| frickler | dcapone2004: run "sasldblistusers2" in the nova-libvirt container and compare to "hostname -f" | 12:02 |
| dcapone2004 | but they are not fqdn ... and they only the hostnames with the individual server ips, nothing for the internal vip | 12:03 |
| dcapone2004 | ok, i will check that in a little bit...just started a reconfigure using source vs binary | 12:04 |
| dcapone2004 | so think the source of the issue is found ... running the command yields "listusers failed" | 12:07 |
| dcapone2004 | same response running as root or nova within the libvirt container | 12:10 |
| opendevreview | Radosław Piliszek proposed openstack/kolla-ansible master: Set any_error_fatal true for gather-facts https://review.opendev.org/c/openstack/kolla-ansible/+/805174 | 12:17 |
| opendevreview | Radosław Piliszek proposed openstack/kolla-ansible master: Allow setting any_errors_fatal true for gather-facts https://review.opendev.org/c/openstack/kolla-ansible/+/805174 | 12:17 |
| dcapone2004 | ok after some conf file digging....i ran sasldblistusers2 - /etc/libvirt/passwd.db and that returned a user | 12:17 |
| dcapone2004 | the hostname matched and matches with each host the container is deployed on.....HOWEVER, the password in the file is showing" userPassword" instead of the generated pw in passwords.yml | 12:19 |
| opendevreview | Merged openstack/kayobe stable/wallaby: Bump stackhpc.drac role https://review.opendev.org/c/openstack/kayobe/+/839699 | 12:39 |
| opendevreview | Merged openstack/kolla-ansible master: Skip /etc/kolla/globals.d on host cleanup https://review.opendev.org/c/openstack/kolla-ansible/+/839769 | 12:43 |
| *** amoralej is now known as amoralej|lunch | 12:48 | |
| opendevreview | Sven Kieske proposed openstack/kolla-ansible stable/ussuri: fix missing lang env in curator crontab https://review.opendev.org/c/openstack/kolla-ansible/+/839912 | 12:50 |
| dcapone2004 | ok, so just reviewed sasldblistusers2 and learned that userPassword is always returned so that the PW is not actually displayed ... so that isn't the issue either | 13:08 |
| dcapone2004 | frickler: libvirt logs are showing "internal error: cannot list SASL mechanisms" ... any ideas on what might cause this? | 13:14 |
| frickler | dcapone2004: just guessing, it might be caused by some pkg missing in the container. which os are you using? | 13:15 |
| frickler | once mgoddard has finished switching into foreground, they might know more ;) | 13:16 |
| mnasiadka | hrw, frickler, yoctozepto, mgoddard: I'm off whole next week | 13:18 |
| hrw | ok | 13:19 |
| dcapone2004 | centos 8....I'm working through the upgrades to get to upgrade everything to centos stream | 13:20 |
| frickler | dcapone2004: maybe you'll want to disable libvirt sasl then until you finish the upgrade | 13:22 |
| frickler | mnasiadka: have fun, hope it's for recreation instead of "only" recovery | 13:23 |
| opendevreview | Verification of a change to openstack/kayobe stable/xena failed: Fix Bifrost inspection through DHCP-relay https://review.opendev.org/c/openstack/kayobe/+/839884 | 13:23 |
| dcapone2004 | frickler: do you know off hand where that option is? I must have misread the release notes for victoria as I thought SASL became required or I would have done that :-) | 13:24 |
| frickler | dcapone2004: "libvirt_enable_sasl: false" in /etc/kolla/globals.yml or whereever you have your kolla config | 13:26 |
| watz | SvenKieske: i may be interested | 13:26 |
| dcapone2004 | frickler: thanks | 13:28 |
| *** amoralej|lunch is now known as amoralej | 13:32 | |
| kevko | guys, I was trying upgrade ussuri on ubuntu 18.04 to ubuntu 20.04 for our customer on my testing env and I met big problem :( ..missing capability ss after upgrade in libvirt ..so live migrate not working ..and i have to reboot instance to fix this issue ... | 13:35 |
| kevko | cat /proc/cpuinfo flags: missing ss ..so there is incompatibility and live-migration will fail .. this commit https://github.com/torvalds/linux/commit/1e03bff3600101bd9158d005e4313132e55bdec8 | 13:36 |
| kevko | I hope they will have CPU which is not affected ...but I am just curious ..how can be this workarounded ? | 13:37 |
| kevko | yoctozepto, hrw, frickler ^^ ? | 13:37 |
| dcapone2004 | frickler: the libvirt_enable_sasl: false option isn't listed int he globals.yml with KA victoria package | 13:53 |
| dcapone2004 | frickler: probably why I couldn't find it and assumed SASL was required, can I just add that line anywhere in globals.yml? | 13:53 |
| mnasiadka | dcapone2004: yes you can | 13:58 |
| dcapone2004 | thx | 14:01 |
| frickler | hrw: just a weird friday afternoon idea: maybe as intermediate solution we want to do split images? c9s/jammy for openstack and c8s/focal for other stuff that still has open external deps? | 14:14 |
| *** amoralej is now known as amoralej|off | 14:24 | |
| dcapone2004 | since I am here and I have some attention....upgrade procedure to move to CentOS Stream from CentOS 8....I obviously am in the process of moving to victoria, 8 and stream are supported...so next step is updating the containers to use Stream and then updating the host OSes to Stream, then upgrading to Wallaby and beyond, correct? | 14:31 |
| SvenKieske | frickler: regarding split images: from an operational point of view I'd like to avoid that: building twice the images, checking CVEs etc.. I can understand the desire though, but it would add twice the amount of work for some things. | 14:41 |
| SvenKieske | watz: I sent you a private message about the event :) | 14:43 |
| opendevreview | Merged openstack/kayobe stable/yoga: Fix Bifrost inspection through DHCP-relay https://review.opendev.org/c/openstack/kayobe/+/839883 | 14:54 |
| opendevreview | Merged openstack/kolla-ansible master: [bifrost] add bifrost_deploy_verbosity parameter https://review.opendev.org/c/openstack/kolla-ansible/+/839133 | 14:57 |
| dcapone2004 | cd /etc/kolla | 15:16 |
| hrw | frickler: prefer not to tbh | 15:20 |
| hrw | dcapone2004: looks sane | 15:21 |
| dcapone2004 | hrw: set openstack_tag_suffix: "-centos8s" in globals.yml and run reconfigure to convert to Stream containers...no other changes needed? | 15:31 |
| hrw | dcapone2004: iirc. but I never did that | 15:31 |
| * hrw -> weekend | 15:31 | |
| opendevreview | Mark Goddard proposed openstack/kayobe master: ironic: Set MTU on provisioning and cleaning Neutron networks https://review.opendev.org/c/openstack/kayobe/+/839965 | 16:15 |
| opendevreview | Vladislav Belogrudov proposed openstack/kolla-ansible stable/yoga: Add doc fix for all-in-one in venv https://review.opendev.org/c/openstack/kolla-ansible/+/839916 | 16:20 |
| opendevreview | Vladislav Belogrudov proposed openstack/kolla-ansible stable/yoga: Skip /etc/kolla/globals.d on host cleanup https://review.opendev.org/c/openstack/kolla-ansible/+/839917 | 16:22 |
| opendevreview | Vladislav Belogrudov proposed openstack/kolla stable/yoga: Add multipath to cinder-volume https://review.opendev.org/c/openstack/kolla/+/839918 | 16:24 |
| opendevreview | Vladislav Belogrudov proposed openstack/kolla stable/yoga: Add multipath to cinder-volume https://review.opendev.org/c/openstack/kolla/+/839918 | 16:25 |
| opendevreview | Merged openstack/kayobe stable/yoga: kolla_passwords: add no_log for password overrides https://review.opendev.org/c/openstack/kayobe/+/839744 | 16:46 |
| opendevreview | Merged openstack/kayobe stable/xena: kolla_passwords: add no_log for password overrides https://review.opendev.org/c/openstack/kayobe/+/839745 | 16:46 |
| opendevreview | Merged openstack/kayobe stable/wallaby: kolla_passwords: add no_log for password overrides https://review.opendev.org/c/openstack/kayobe/+/839746 | 16:46 |
| opendevreview | Merged openstack/kayobe stable/victoria: kolla_passwords: add no_log for password overrides https://review.opendev.org/c/openstack/kayobe/+/839747 | 16:46 |
| greatgatsby_ | Hi. Have install openstack with kolla-ansible. Just starting to get octavia setup. Seems by default a vxlan network will be created for the management network. Is this preferred, or should I manually configure a vlan provider network? | 17:50 |
| johnsom | No load balanced traffic crosses the management network, it's just command/control/stats. | 17:57 |
| opendevreview | Merged openstack/kayobe stable/xena: Fix Bifrost inspection through DHCP-relay https://review.opendev.org/c/openstack/kayobe/+/839884 | 18:04 |
| greatgatsby_ | johnsom, thanks for the reply. Apologies, I'm still quite new to openstack and getting a bit lost in the kolla-ansible octavia docs, where they say "Typically a VLAN provider network is used". | 18:05 |
| johnsom | I am not a kolla expert either, but I can talk to Octavia. In general, starting out I would stick to the defaults in kolla to keep it simple. | 18:08 |
| greatgatsby_ | great, thanks | 18:12 |
| opendevreview | James Kirsch proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs https://review.opendev.org/c/openstack/kolla-ansible/+/741340 | 18:29 |
| jamesbenson | oklhost_: We open sources how we deploy openstack at our university as well. We have our own gitlab runner locally, so we literally just run a job ands it deploys here. https://gitlab.com/utsa-ics/osias | 21:33 |
| jamesbenson | This includes installing a fresh OS using MaaS | 21:33 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!