Monday, 2022-09-26

*** ysandeep|out is now known as ysandeep05:42
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/yoga: Include install_method variables for openrc  https://review.opendev.org/c/openstack/openstack-ansible/+/85916807:20
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/xena: Include install_method variables for openrc  https://review.opendev.org/c/openstack/openstack-ansible/+/85916907:20
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible stable/wallaby: Include install_method variables for openrc  https://review.opendev.org/c/openstack/openstack-ansible/+/85923007:20
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-python_venv_build stable/yoga: Change default value for venv_wheel_build_enable  https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/85923107:23
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_keystone stable/yoga: Bootstrap when running against last backend  https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/85923207:29
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_keystone stable/yoga: Bootstrap when running against last backend  https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/85923207:31
dokeeffe85Morning, still a little stuck with the networking on OSA, internal nets are working fine but I can't get an external network working. I can see the ovs bridge br-vlan and it has the right interface but when I spin up a vm and look at the ports they all have br-int as their bridge https://paste.openstack.org/show/btlS1NdYgvzKT4okkwwV/08:20
noonedeadpunkdokeeffe85: so there are 2 ways of getting external networking - 1. through l3 router and floarting ips, 2. pass public network to each compute08:25
*** ysandeep is now known as ysandeep|afk08:25
noonedeadpunkIf you're trying to get public network passed directly to a vm, then it's option 2. And it quite depends on how you have created the network on neutron08:26
noonedeadpunkas then you should explicitly provide network type to be flat for vlan and pass correct vlanid (if it's vlan)08:27
dokeeffe85Ah ok, it's the second way. So it should be a flat network not a vlan. Ok let me re do that so. Thanks08:30
dokeeffe85noonedeadpunk thanks a million, all sorted08:38
opendevreviewMerged openstack/ansible-role-systemd_mount master: Remove redundant vars line  https://review.opendev.org/c/openstack/ansible-role-systemd_mount/+/85786108:54
opendevreviewMerged openstack/ansible-role-uwsgi master: Remove redundant vars line  https://review.opendev.org/c/openstack/ansible-role-uwsgi/+/85787608:59
opendevreviewMerged openstack/openstack-ansible-os_masakari master: Remove redundant vars line  https://review.opendev.org/c/openstack/openstack-ansible-os_masakari/+/85765309:05
opendevreviewMerged openstack/openstack-ansible-rabbitmq_server master: Remove redundant vars line  https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/85765809:06
opendevreviewMerged openstack/openstack-ansible-os_cloudkitty master: Remove redundant vars line  https://review.opendev.org/c/openstack/openstack-ansible-os_cloudkitty/+/85765009:07
opendevreviewMerged openstack/openstack-ansible-rsyslog_client master: Remove redundant vars line  https://review.opendev.org/c/openstack/openstack-ansible-rsyslog_client/+/85783709:09
opendevreviewMerged openstack/openstack-ansible-os_adjutant master: Refactor use of include_vars  https://review.opendev.org/c/openstack/openstack-ansible-os_adjutant/+/85782709:17
*** ysandeep|afk is now known as ysandeep09:50
opendevreviewMerged openstack/openstack-ansible-ops master: Correct OVNL3RouterPlugin path  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/85656909:51
admin1dokeeffe85, here is a doc for the 2nd way => https://www.openstackfaq.com/openstack-add-direct-attached-dhcp-ip/. .. then you just select the network like you select private, but you get direct (public) ips 09:57
admin1not an osa question, but just asking if people know: is it possible to update/change  a project from one domain to another ? 10:00
noonedeadpunkI don't really think so.10:21
noonedeadpunkBut you totally can grant permissions to user from another domain to project in different domain10:21
noonedeadpunkOr hacking DB10:22
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Fix cinder-volume AIO ceph scenario  https://review.opendev.org/c/openstack/openstack-ansible/+/85926310:54
admin1i will try it in hacking a db :) 11:06
opendevreviewMerged openstack/openstack-ansible-os_ironic master: Replace pxe_append_params with kernel_pxe_params in ironic.conf  https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/85659011:06
*** ysandeep is now known as ysandeep|afk11:20
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Avoid ovs restart during package upgrade  https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/85765211:21
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Avoid ovs restart during package upgrade  https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/85765211:21
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Avoid ovs restart during package upgrade  https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/85765211:21
kleini_Is it possible with config_overrides to remove some option from an ini config file section? Providing no value gives key = None but I would like to remove key.11:24
*** kleini_ is now known as kleini11:24
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-galera_server master: Use policy_rc_d attribute instead of copy  https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/85927211:28
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-lxc_hosts master: Use policy_rc_d attribute instead of copy  https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/85927311:31
noonedeadpunkkleini: I think it's not possible right now. But it's might be good thing to implement indeed (though quite tricky)11:32
noonedeadpunkbest you can do is define option to empty string11:33
noonedeadpunkHighly likely oslo.config will interpret that as variable being not defined11:34
noonedeadpunkBut can't remember for sure11:34
*** ysandeep|afk is now known as ysandeep11:38
noonedeadpunk(but can recall some discussions about that)11:40
noonedeadpunk(and I think we edned up with setting empty string is quite same as not define at all for oslo)11:41
*** ysandeep is now known as ysandeep|afk12:40
kleinimy current case is about ansible-role-systemd_networkd and I would like to configure the Match section with a MACAddress and not with a Name. But there seems currently no possible solution in my case: https://opendev.org/openstack/ansible-role-systemd_networkd/src/branch/master/templates/systemd-network.j2#L412:43
kleiniwill put that on my TODO list12:45
mgariepyhttps://opendev.org/openstack/ansible-config_template/src/branch/master/library/config_template#L52-L5712:53
mgariepykleini, isn't that suppose to clean up none value from the override?12:54
kleinishould be worth a try12:57
*** ysandeep|afk is now known as ysandeep13:08
kleinihmm, default of ignore_none_type is true. I set it to true but I still get "Name = None" in my ens3.network file. So for me this option seems to be broken.13:11
mgariepyhmm :/ sad/13:15
kleiniGot it working. My problem was, that I define input to ansible-role-systemd_networkd as yaml. And the equivalent of None in Yaml is "Name: {}"13:17
kleinivery nice!13:17
mgariepycool13:18
*** ysandeep is now known as ysandeep|out14:10
noonedeadpunkit's worth documenting somewhere....14:12
noonedeadpunknot too obvious I would say14:12
kleinishould I document that within the config_template collection or systemd_networkd role?14:23
kleinimaybe I can add both. my example case for systemd_networkd and something general for config_template.14:23
noonedeadpunkkleini: I think it make sense to add it to integrarted repo14:42
noonedeadpunklike https://opendev.org/openstack/openstack-ansible/src/branch/master/doc/source/reference/configuration/using-overrides.rst14:43
psyminI've recently installed openstack ansible aio on a rocky machine and I have a few rather novice questions.  For example, I'm not sure how to ssh to the box any more.15:09
*** dviroel is now known as dviroel|lunch15:10
noonedeadpunkpsymin: well, aio should not drop your keypair from machine. but I do recall such behaviour in past, and it was related to either some networking issue (when aio-provisioned networks were intersecting with actuall network on machine) 15:12
noonedeadpunkas we do hardcode nets, so you can get quite unlucky15:12
noonedeadpunkbut other then that you should be able to ssh to VM as ususal15:16
noonedeadpunk*usual15:16
psyminIt appears root login w/ password was disabled after the aio install.  Resolved.15:42
psyminFor the real install I will use keys.15:42
noonedeadpunkoh yes. it's effect of hardening playbook :)15:46
noonedeadpunkfor some reason I didn't assume that you had root login with password, sorry15:46
noonedeadpunkyou can disable this behaviour if you want to15:47
psyminIt is a valid assumption to make.  I won't be using aio for production, just getting a feel for the system.  After this seems to work we'll be trying it on three baremetal machines.15:47
noonedeadpunkyou can set `security_sshd_permit_root_login: yes` if needed 15:48
noonedeadpunk(in user_variables.yml)15:48
psyminahh in the ansible playbook, cool.  It isn't needed yet :)  I'll use the keys for the next step of install testing.15:49
psyminIs there a good article I could read for learning how to use the default horizon interface in an aio install?15:50
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Fix cinder-volume AIO ceph scenario  https://review.opendev.org/c/openstack/openstack-ansible/+/85926315:58
psyminmy next issue is that I'm trying to test a 32gb qcow2 file to verify it can work, but "copy image to volume:Image used for creating volume exceeds available space."16:04
psyminso I need to learn how to add existing block devices for space, or something16:04
noonedeadpunkwell, for aio we create loop devices and I think we assume having 100gb drive (as it's what we have in CI)16:06
noonedeadpunkI think we use /dev/sdb if any is present, but I need to double-check16:07
psyminall block devices on this baremetal are 1.8tb, I do see some loop devices created /dev/loop2 through /dev/loop6  .. /dev/loop6 is only 11gigs and mapped to /var/lib/machines16:09
noonedeadpunkyou can use `export BOOTSTRAP_OPTS="bootstrap_host_data_disk_device=sdb"` before running bootstrap_aio.sh16:10
noonedeadpunkah, ok, so if it's root device.... Let me check how we do create loop drives16:11
noonedeadpunkwe have variable bootstrap_host_loopback_cinder_size for cinder loop device size at very least16:11
psymingotcha16:12
noonedeadpunkcheck this out https://opendev.org/openstack/openstack-ansible/src/branch/master/tests/roles/bootstrap-host/defaults/main.yml#L46-L10016:12
psyminlvdisplay, pvdisplay, df -h  https://pastebin.com/raw/775NNPVk16:14
noonedeadpunkok, yes, so loop device is big enough indeed16:14
noonedeadpunkwait, how have you created volume from image?16:15
noonedeadpunkAs if you have not specified min_disk for the image, you would need to provide disk size explicitly16:15
noonedeadpunkand you do have minimal disk created16:16
psyminI uploaded the qcow file using "create image" under "compute" in horizon.  then I "create volume" under that image, changed the default size from 32gb to 33gb for a buffer, and clickec "create volume"16:17
psyminIt is very likely I'm taking the wrong steps.16:17
psyminif there is a different channel I should be asking this novice questions in, please let me know16:18
noonedeadpunkok, so you're using horizon. when you're uploading image - you can define properties for it. Among them you can define minimal disk. Also when creating volume you can set volume size. 16:19
noonedeadpunkbtw, you've isntalled Yoga?16:19
psyminYep, yoga, rocky 8.  Branch 25.1.0 I think.16:19
noonedeadpunkAs we were recently reported that horizon appears broken on master16:19
psyminIf there is a different path I should be taking I'm all ears.  Rocky 9?16:20
*** dviroel|lunch is now known as dviroel16:20
noonedeadpunkpsymin: nah, I was jsut wondering :) Rocky 9 is WIP and we want to backport it's support to Yoga quite soon16:21
noonedeadpunkpsymin: I guess your issue at the moment is more about data you've filled in either when creating volume or when umploading image16:21
noonedeadpunk*uploading16:21
noonedeadpunkpsymin: are you familiar with openstack cli?16:22
psyminnoonedeadpunk, not at all familiar.  I can learn though.  16:22
psyminPerhaps it would be wise for me to configure user_variables.yml to use the eight 1.8 tb drives, and reinstall?16:23
noonedeadpunkok, so it's pre-installed inside utility container with aio16:23
psyminis there a way to ssh to that container from the host?16:24
noonedeadpunkso you can do lxc-ls to list container and then lxc-attach -n aio_utility_(smth-smth)16:24
noonedeadpunkonce you're in - run `source /root/openrc`16:24
noonedeadpunkthat will give CLI admin privileges16:24
noonedeadpunkso out of interest - `openstack image list` `openstack image show $UUID` of your image16:25
noonedeadpunkAnd you can create volume with command `openstack volume create --size 100 --image $UUID somename --bootable`16:26
noonedeadpunksize is in GB16:26
psyminahh, in the utility container is where I can use the openstack command, thank you so much.16:27
noonedeadpunkwell, you can install python-openstackclient anywhere16:28
noonedeadpunkbut it's pre-installed there16:28
noonedeadpunkutility container also does have .my.cnf so you can access DB from there as well16:28
psyminstatus is "error" in `openstack volume list` as well as in horizon after creating the volume from the image.16:31
psyminI used 100 even though the image is 3216:32
psyminwould a path of less resistance be to try using the three baremetal servers as a test as opposed to aio?16:33
noonedeadpunkand what is `openstack volume show` returns you as error reason?16:37
noonedeadpunkalso what volume backend do you want to use?16:37
psyminafter learning more, I hope to use ceph16:38
psyminI don't see a reason with "openstack volume list <id>", just "status: error", but in horizon "Image used for creating volume exceeds available space."16:39
noonedeadpunkWell, I would assume to spawn aio with ceph scenario but found out today that it's currently broken. I pushed https://review.opendev.org/c/openstack/openstack-ansible/+/859263 in order to fix it16:39
noonedeadpunk"openstack volume show <id>"16:40
noonedeadpunkyou meant?16:40
psyminhttps://pastebin.com/raw/tA2LBRHG16:40
psyminyep, apologies16:40
noonedeadpunkopenstack volume service list?16:41
noonedeadpunkeventually I can assume that we've got cinder jsut broken in aio....16:41
noonedeadpunkas appeared we were not testing cinder properly16:41
psyminhttps://pastebin.com/raw/0HAmgpFN16:41
noonedeadpunkhm, but it looks quite healthy16:42
psyminI'd assume that the error is on my end rather than with your tools16:42
noonedeadpunkyou can enter cinder-volume container and check logs with journalctl16:43
noonedeadpunkBut i'm not sure if error happens on cinder or glance side...16:43
noonedeadpunkAnyway you will know more about how to troubleshoot deployments :)16:43
noonedeadpunkeventually I can recall some troubles with iscsi configuration for aio as well, but well. never had time to dig into that16:50
noonedeadpunkpsymin: though, you actually don't need volume to spawn a VM from the image16:50
noonedeadpunkYou can create server directly from the image. Despite it's not best way to do that - but at least you will be able to test your image right away and see if it works16:51
noonedeadpunkand sort out volume thing later16:51
noonedeadpunkyou can do that from horizon as well, on server creation page you will need to select No on question if create new volume16:52
psyminI am seeing issues in journalctl for cinder.  More context: this baremetal had eno1 for a nic that would start onboot.  After installing aio the nic changed to eth0.  This caused some issue.  This mornign I set eth0 to come up on boot and rebooted.16:52
noonedeadpunkyeah, I guess we're unifying things with AIO and mess with networking quite a lot16:54
noonedeadpunkwe mostly use  aio for ci/cd, so need to ensure state let's say16:55
psyminperhaps I should install rocky 8 on these three machines, get them online, create a better user_variables.yml file, and test that way?16:56
noonedeadpunk(and openstack_user_config)16:58
noonedeadpunkwell, it depends on goals. that's totally worth doing for production, yes16:58
noonedeadpunkaio totally not designed for any real usage16:59
noonedeadpunkmore for checking out concepts and quickly check out some things16:59
noonedeadpunkor run CI16:59
psyminEnd goal is for production locally for one organization, no untrusted guests.16:59
noonedeadpunkso at the end you will still need to create deployment from scratch )17:00
noonedeadpunkcan you paste errors you've seen in cinder containers?17:01
psyminSure, they may not be relevant to the immediate issue though, since there were networking issues that caused haproxy to fail, for example.17:02
noonedeadpunkah17:02
noonedeadpunkyou can actually do `journalctl -f -u cinder-*` and try to reproduce issue. Command will follow current logs 17:03
noonedeadpunkaio does enable debug by default, so you might want to redirect output...17:04
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Bump ansible-core version to 2.13.4  https://review.opendev.org/c/openstack/openstack-ansible/+/85750617:14
*** dviroel is now known as dviroel|dr_appt17:20
psyminpython3.6 is listed in the journalctl (and in the horizon details traceback).  Should it be 3.8?17:34
psyminhttps://pastebin.com/raw/Jse0TuTG17:35
noonedeadpunkwell yes, I'd assume it to be 3.817:41
noonedeadpunkthough it's unlikely root cause of the issue you see17:42
noonedeadpunkpsymin: I will spawn up Rocky tomorrow in aio to check what's wrong with python versions there17:48
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Bump ansible-core version to 2.13.4  https://review.opendev.org/c/openstack/openstack-ansible/+/85750617:50
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-python_venv_build stable/yoga: Change default value for venv_wheel_build_enable  https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/85923118:02
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Fix cinder-volume AIO ceph scenario  https://review.opendev.org/c/openstack/openstack-ansible/+/85926318:30
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Fix cinder-volume AIO ceph scenario  https://review.opendev.org/c/openstack/openstack-ansible/+/85926318:36
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Extend NFS scenario to support cinder  https://review.opendev.org/c/openstack/openstack-ansible/+/85933618:40
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_cinder master: Add nfs and ceph jobs to templates  https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/85933718:43
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Run test_minimum_basic_scenario test for cinder  https://review.opendev.org/c/openstack/openstack-ansible/+/85933919:19
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_cinder master: Add nfs and ceph jobs to templates  https://review.opendev.org/c/openstack/openstack-ansible-os_cinder/+/85933719:20
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Fix cinder-volume AIO ceph scenario  https://review.opendev.org/c/openstack/openstack-ansible/+/85926319:31
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Extend NFS scenario to support cinder  https://review.opendev.org/c/openstack/openstack-ansible/+/85933619:32
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Run test_minimum_basic_scenario test for cinder  https://review.opendev.org/c/openstack/openstack-ansible/+/85933919:32
prometheanfireis there something 'decided upon' for monitoring an osa deploy?  I imagine the elk metrics from osa-ops will be good for metrics but not alerting, at least at first20:12
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Fix cinder-volume AIO ceph scenario  https://review.opendev.org/c/openstack/openstack-ansible/+/85926321:01
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Extend NFS scenario to support cinder  https://review.opendev.org/c/openstack/openstack-ansible/+/85933621:01
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Run test_minimum_basic_scenario test for cinder  https://review.opendev.org/c/openstack/openstack-ansible/+/85933921:01
noonedeadpunkprometheanfire: nah, not really. Well, we do have some prometheus playbook in ops but I have really no idea when and if anybody ever used it https://opendev.org/openstack/openstack-ansible-ops/src/branch/master/prometheus21:08
noonedeadpunkand it really noop rather then any real thing imo21:09
noonedeadpunkbut I guess for monitoring it really pretty much depends as it's quite spread across some nrpe/zabbix/prometheus so everybody use own thing as a result21:10
prometheanfireya, I saw the prometheus thing there, not updated in 4 years22:55

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!