| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Just test something https://review.opendev.org/c/openstack/kolla-ansible/+/893609 | 05:49 |
|---|---|---|
| yusufgungor | Hi everyone, kolla nova_compute image has embedded /etc/nvme/hostnqn and /etc/nvme/hostid files. This files automatically created after nvmi-cli package installation. Using same hostnqn value for multiple host brokes the nvme connections. nova does not try to create this files if does not exist and gives error like below: | 06:53 |
| yusufgungor | Command: nvme show-hostnqn | 06:53 |
| yusufgungor | Exit code: 254 | 06:53 |
| yusufgungor | Stdout: '' | 06:53 |
| yusufgungor | Stderr: 'hostnqn is not available -- use nvme gen-hostnqn\n' | 06:53 |
| yusufgungor | 2023-09-06 18:39:51.456 144 WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: Unexpected error while running command. | 06:53 |
| yusufgungor | This break the purestorage connection which is supported by kolla-ansible and not works out of box. | 06:53 |
| yusufgungor | What do you think about as a solution? Should kolla remove this files after installing nvme-cli and nova create them again? Or kolla should have an entrypoint cmd in containers to create nqn files to let them unique? | 06:53 |
| *** ralonsoh_away is now known as ralonsoh | 07:09 | |
| mnasiadka | yusufgungor: if nova is not creating them/updating them, then we should make them unique - question if they should not be static on given hostname (e.g. we should not recreate them on every container restart - just manage that in kolla-ansible) | 07:12 |
| mnasiadka | yusufgungor: can you please raise a bug in launchpad? | 07:12 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Just test something https://review.opendev.org/c/openstack/kolla-ansible/+/893609 | 07:12 |
| yusufgungor | Hi @mnasiadka thanks for you reply. Bug should be created on kolla or kolla-ansible? | 07:17 |
| kevko | yusufgungor: hmm, it's only warning ..if i remember correctly ..this command failed and logged error as warning ..but there are another 2 if/else statements where ID is generated by something else | 07:20 |
| kevko | yusufgungor: and it works ... no ? | 07:21 |
| yusufgungor | @kevko i have tried to removing the hostnqn file but unfortunately nova does not recreate it. Logs can be seen from here: https://paste.opendev.org/show/bREvNqcrj0rHtSU9IEeN/ | 07:24 |
| sahid | o? | 07:44 |
| sahid | o/ | 07:44 |
| sahid | quick question, if I need when I enable osprofiler to set option https://github.com/openstack/osprofiler/blob/master/osprofiler/opts.py#L220 | 07:44 |
| sahid | should I have to propose a patch in kolla to give that ability or is there any native solution to do that? | 07:45 |
| kevko | sahid: osprofiler is just block of config in some service config , isn't it ? (don't remember ....) | 07:57 |
| kevko | yusufgungor: give me few minutes please | 07:57 |
| sahid | kevko: yes right | 07:57 |
| sahid | a section [profiler] is added | 07:58 |
| kevko | sahid: in that case you can use config override - that means you can create some config file and place to kolla configuration dir and include block of config [profiler] foo = bar inside the file .... result will be the combination of what kolla is rendering and what you will add ... | 08:00 |
| kevko | sahid: for example /etc/kolla/config/nova.conf | 08:00 |
| sahid | that looks to be exactly what I need | 08:01 |
| sahid | thanks a lot ! | 08:01 |
| kevko | sahid: you can also place that config to special config override /etc/kolla/config/global.conf | 08:02 |
| sahid | yes I just noticed the doc related ++ kevko | 08:02 |
| kevko | sahid: this means that profiler block of config will be merged for all services | 08:02 |
| kevko | sahid: yes - this is it -> https://docs.openstack.org/kolla-ansible/latest/admin/advanced-configuration.html | 08:02 |
| kevko | sahid: you're welcome | 08:03 |
| kevko | yusufgungor: this block of code is generating warnings/error - https://github.com/openstack/os-brick/blob/bb730f37cd40edfc09688a14c1f3da05003383fa/os_brick/initiator/connectors/nvmeof.py#L763-L786C24 | 08:06 |
| opendevreview | Merged openstack/kolla-ansible stable/2023.1: ovn: Improve clustering https://review.opendev.org/c/openstack/kolla-ansible/+/893770 | 08:10 |
| kevko | yusufgungor: what did you get when you tried to run 'nvme gen-hostnqn 'in nova container ? | 08:12 |
| mmalchuk | good morning Kolla o/ | 08:15 |
| kevko | \o/ | 08:17 |
| yusufgungor | @kevko gen-hostnqn command works. | 08:19 |
| yusufgungor | (nova-compute)[nova@my-compute-01 /]$ nvme gen-hostnqn | 08:19 |
| yusufgungor | nqn.2014-08.org.nvmexpress:uuid:9330688a-e4a8-41ad-b780-1fadb46618f7 | 08:19 |
| yusufgungor | (nova-compute)[nova@my-compute-01 /]$ | 08:19 |
| yusufgungor | I have deleted the hostnqn file to see if nova-compute recreate it. But as you can see from the warning log, it does not try to regenerate, it only tries show-hostnqn. Stderr states that hostnqn is not available and use gen-hostnqn | 08:19 |
| yusufgungor | Command: nvme show-hostnqn | 08:19 |
| yusufgungor | Exit code: 254 | 08:19 |
| yusufgungor | Stdout: '' | 08:19 |
| yusufgungor | Stderr: 'hostnqn is not available -- use nvme gen-hostnqn\n' | 08:19 |
| yusufgungor | 2023-09-06 18:39:51.456 144 WARNING os_brick.privileged.nvmeof [-] Could not generate host nqn: Unexpected error while running command. | 08:19 |
| opendevreview | Merged openstack/kolla-ansible stable/zed: ovn: Improve clustering https://review.opendev.org/c/openstack/kolla-ansible/+/893771 | 08:20 |
| opendevreview | Merged openstack/kolla-ansible stable/yoga: ovn: Improve clustering https://review.opendev.org/c/openstack/kolla-ansible/+/893772 | 08:20 |
| sahid | kevko: actually I was wrong... | 08:21 |
| sahid | we want to set option is the global config file of osprofiler itself | 08:22 |
| kevko | yusufgungor: i see, checking the code | 08:24 |
| kevko | yusufgungor: version of nvmecli ? | 08:25 |
| kevko | yusufgungor: i would add some debug logs to this function and see what is happening there -> https://github.com/openstack/os-brick/blob/bb730f37cd40edfc09688a14c1f3da05003383fa/os_brick/privileged/nvmeof.py#L33-L76C20 | 08:35 |
| kevko | yusufgungor: because if nvme gen-hostnqn is working inside nova container ...it can be rootwrap ..or something else ...just debug which line causing exception .. | 08:36 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Just test something https://review.opendev.org/c/openstack/kolla-ansible/+/893609 | 08:45 |
| yusufgungor | @kevko | 09:02 |
| yusufgungor | $ nvme --version | 09:02 |
| yusufgungor | nvme version 1.16 | 09:02 |
| yusufgungor | i think it is not trying gen-hostnqn at all, it only tries to show-hostnqn. I have deleted the hostnqn file on purpose for simulating to see if nova_compute create a new nqn file. My solution idea was deleting the default hostnqn file after building container image and let the nova compute creates a new one to let it unique for every runnin nova compute container | 09:02 |
| kevko | yusufgungor: check the code | 09:05 |
| yusufgungor | @kevko yes there exist create_hostnqn() method in os-brick nvmeof.py file but i am not sure this method is called from nova | 09:07 |
| kevko | yusufgungor: it is, turn on debug | 09:07 |
| kevko | yusufgungor: you should see exact LOG.debug | 09:08 |
| yusufgungor | @kevko ok i'll check it, thanks 🙏 | 09:08 |
| kevko | yusufgungor: also in your log it is WARNING os_brick.privileged.nvmeof ....it just means | 09:08 |
| kevko | os_brick ....etc | 09:08 |
| kevko | yusufgungor: after debug you will be able to see bigger context - i mean ..you will see where it will fail ... | 09:09 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: add block support to validate-all-file.py https://review.opendev.org/c/openstack/kolla-ansible/+/894155 | 10:02 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: service-*: run docker exec commands using become https://review.opendev.org/c/openstack/kolla-ansible/+/894158 | 10:07 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: add block support to validate-all-file.py https://review.opendev.org/c/openstack/kolla-ansible/+/894155 | 10:08 |
| opendevreview | Maksim Malchuk proposed openstack/kayobe stable/zed: Pass through kolla inventories as is https://review.opendev.org/c/openstack/kayobe/+/893688 | 10:53 |
| opendevreview | Maksim Malchuk proposed openstack/kayobe stable/yoga: Pass through kolla inventories as is https://review.opendev.org/c/openstack/kayobe/+/894172 | 11:47 |
| opendevreview | Maksim Malchuk proposed openstack/kayobe stable/xena: Pass through kolla inventories as is https://review.opendev.org/c/openstack/kayobe/+/894173 | 11:49 |
| opendevreview | Maksim Malchuk proposed openstack/kayobe stable/2023.1: Use merge_configs and merge_yaml to generate Kolla custom config https://review.opendev.org/c/openstack/kayobe/+/894174 | 11:51 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add support of podman deployment https://review.opendev.org/c/openstack/kolla-ansible/+/799229 | 11:54 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix kolla_logs mount point https://review.opendev.org/c/openstack/kolla-ansible/+/894175 | 11:54 |
| opendevreview | Maksim Malchuk proposed openstack/kayobe stable/zed: Use merge_configs and merge_yaml to generate Kolla custom config https://review.opendev.org/c/openstack/kayobe/+/894176 | 11:55 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: add block support to validate-all-file.py https://review.opendev.org/c/openstack/kolla-ansible/+/894155 | 12:03 |
| mmalchuk | fungi /cle | 12:03 |
| mmalchuk | ups... sorry | 12:03 |
| mmalchuk | wrong window) | 12:04 |
| opendevreview | Maksim Malchuk proposed openstack/kayobe stable/2023.1: Kayobe environment dependencies https://review.opendev.org/c/openstack/kayobe/+/894178 | 12:11 |
| opendevreview | Maksim Malchuk proposed openstack/kayobe stable/zed: Kayobe environment dependencies https://review.opendev.org/c/openstack/kayobe/+/894180 | 12:17 |
| mhiner | Hello, can you clear up the code smell situation in podman patch in a-c-k? In that patch, do we want to keep the current approach from docker or go ahead with the "better" one? | 12:24 |
| kevko | mhiner: a-c-k ? | 12:25 |
| kevko | mhiner: which better one ? | 12:25 |
| mhiner | ansible-collection-kolla | 12:26 |
| mhiner | the one proposed here: https://review.opendev.org/c/openstack/ansible-collection-kolla/+/892990 | 12:26 |
| mhiner | I'm asking if I should mirror it in my patch (as per ebbex's comments) or if I should keep the current approach | 12:27 |
| mmalchuk | imho its better not copy bad practice, and refactor the code step by step | 12:29 |
| opendevreview | Christian Berendt proposed openstack/kolla-ansible master: ceilometer: handle custom event_pipeline.yaml as template https://review.opendev.org/c/openstack/kolla-ansible/+/894182 | 12:40 |
| opendevreview | Martin Hiner proposed openstack/ansible-collection-kolla master: Add Podman support https://review.opendev.org/c/openstack/ansible-collection-kolla/+/852240 | 13:07 |
| opendevreview | Martin Hiner proposed openstack/ansible-collection-kolla master: Add Podman support https://review.opendev.org/c/openstack/ansible-collection-kolla/+/852240 | 13:08 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: add block support to validate-all-file.py https://review.opendev.org/c/openstack/kolla-ansible/+/894155 | 13:33 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: add block support to validate-all-file.py https://review.opendev.org/c/openstack/kolla-ansible/+/894155 | 13:36 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: add block support to validate-all-file.py https://review.opendev.org/c/openstack/kolla-ansible/+/894155 | 13:45 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: add block support to validate-all-file.py https://review.opendev.org/c/openstack/kolla-ansible/+/894155 | 13:52 |
| opendevreview | Mark Goddard proposed openstack/kolla-ansible stable/yoga: Fix typo in endpoint influxdb_internal_endpoint variable https://review.opendev.org/c/openstack/kolla-ansible/+/893953 | 14:01 |
| opendevreview | Mark Goddard proposed openstack/kolla-ansible stable/xena: Fix typo in endpoint influxdb_internal_endpoint variable https://review.opendev.org/c/openstack/kolla-ansible/+/893954 | 14:01 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/yoga: [yoga only] followup ovn clustering fix https://review.opendev.org/c/openstack/kolla-ansible/+/894194 | 14:04 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: add block support to validate-all-file.py https://review.opendev.org/c/openstack/kolla-ansible/+/894155 | 14:11 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: add block support to validate-all-file.py https://review.opendev.org/c/openstack/kolla-ansible/+/894155 | 14:17 |
| opendevreview | Bartosz Bezak proposed openstack/kolla-ansible master: CI: add q35 hardware machine type to tests https://review.opendev.org/c/openstack/kolla-ansible/+/894199 | 14:26 |
| opendevreview | Bartosz Bezak proposed openstack/kolla-ansible master: Configure Nova libvirt.num_pcie_ports to 16 by default https://review.opendev.org/c/openstack/kolla-ansible/+/768301 | 14:29 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: add block support to validate-all-file.py https://review.opendev.org/c/openstack/kolla-ansible/+/894155 | 14:34 |
| mmalchuk | mnasiadka please review https://review.opendev.org/q/I28f5d73d414d405d67f5fc92ab371aa2e28a4ce3 | 14:34 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix podman logs https://review.opendev.org/c/openstack/kolla-ansible/+/893187 | 14:49 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Fix kolla_logs mount point https://review.opendev.org/c/openstack/kolla-ansible/+/894175 | 14:49 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add support of podman deployment https://review.opendev.org/c/openstack/kolla-ansible/+/799229 | 14:49 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Just test something https://review.opendev.org/c/openstack/kolla-ansible/+/893609 | 14:49 |
| opendevreview | Bartosz Bezak proposed openstack/kolla-ansible master: CI: add q35 hardware machine type to tests https://review.opendev.org/c/openstack/kolla-ansible/+/894199 | 15:18 |
| opendevreview | Bartosz Bezak proposed openstack/kolla-ansible master: Configure Nova libvirt.num_pcie_ports to 16 by default https://review.opendev.org/c/openstack/kolla-ansible/+/768301 | 15:19 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: add block support to validate-all-file.py https://review.opendev.org/c/openstack/kolla-ansible/+/894155 | 15:40 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/yoga: [yoga only] followup ovn clustering fix https://review.opendev.org/c/openstack/kolla-ansible/+/894194 | 15:49 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: CI: add block support to validate-all-file.py https://review.opendev.org/c/openstack/kolla-ansible/+/894155 | 15:50 |
| opendevreview | Dr. Jens Harbott proposed openstack/kolla-ansible master: DNM: Test octavia amp subnet https://review.opendev.org/c/openstack/kolla-ansible/+/894219 | 18:21 |
| *** dcapone2004 is now known as Guest2161 | 19:30 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!