| *** bauzas_ is now known as bauzas | 00:04 | |
| *** bauzas_ is now known as bauzas | 00:21 | |
| *** bauzas_ is now known as bauzas | 07:28 | |
| opendevreview | Sylvain Bauza proposed openstack/nova master: WIP : Enable an experimental job for testing mtty https://review.opendev.org/c/openstack/nova/+/922140 | 07:33 |
|---|---|---|
| opendevreview | Sylvain Bauza proposed openstack/nova master: WIP : mtty: add devstack compilation for fedora builds https://review.opendev.org/c/openstack/nova/+/918420 | 07:33 |
| sahid | o/ | 07:43 |
| sahid | gibi: morning, regarding https://review.opendev.org/c/openstack/nova/+/921665 - it is not somerhinf that we should fix in placement if this happens? | 07:43 |
| Uggla | Hello sean-k-mooney, can you please have a look the comment https://review.opendev.org/c/openstack/nova/+/833090/comment/82eda0c2_3d07f30a/ | 08:33 |
| bauzas | sean-k-mooney: frickler: do you know why FORCE=yes wasn't passed to compute1 here : https://review.opendev.org/c/openstack/nova/+/922140/6/.zuul.yaml#345 | 09:24 |
| bauzas | it was a global var | 09:24 |
| sean-k-mooney | you have to set it on the sub node too | 09:25 |
| bauzas | https://zuul.opendev.org/t/openstack/build/99ecc0f9124248739e2ec4d1b959cf5e/log/job-output.txt#24841 | 09:25 |
| sean-k-mooney | we run devstack twice indepently when doing multi node so both have diffent local.cons | 09:25 |
| bauzas | even if it was set in vars: | 09:25 |
| bauzas | ? | 09:26 |
| sean-k-mooney | yes | 09:26 |
| sean-k-mooney | even if set in vars | 09:26 |
| sean-k-mooney | look at nova next | 09:27 |
| bauzas | ack | 09:28 |
| bauzas | thanks | 09:28 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: WIP : Enable an experimental job for testing mtty https://review.opendev.org/c/openstack/nova/+/922140 | 09:32 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: WIP : mtty: add devstack compilation for fedora builds https://review.opendev.org/c/openstack/nova/+/918420 | 09:32 |
| opendevreview | Andrew Bonney proposed openstack/nova master: WIP: Work around deleted instances when performing data migration https://review.opendev.org/c/openstack/nova/+/922299 | 10:22 |
| *** bauzas_ is now known as bauzas | 11:57 | |
| *** bauzas_ is now known as bauzas | 12:16 | |
| obrest | Hi! Would it be a good idea by nova to add Intel AMX to the list of features it identifies an publishes traits for in placement? | 12:30 |
| obrest | I just got a couple of sapphire-rapids compute-nodes; and I would like to expose flavors with AMX to my users; but that proved to be a bit difficult since nova is not identifying this trait. | 12:31 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: WIP : Enable an experimental job for testing mtty https://review.opendev.org/c/openstack/nova/+/922140 | 12:55 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: WIP : mtty: add devstack compilation for fedora builds https://review.opendev.org/c/openstack/nova/+/918420 | 12:55 |
| opendevreview | Merged openstack/nova stable/2023.1: [ironic] Partition & use cache for list_instance* https://review.opendev.org/c/openstack/nova/+/922155 | 13:34 |
| elodilles | bauzas: now that you are editing .zuul.yaml: there are some warnings regarding deprecated regex syntax. if that irritates you, here is a fix for that: https://review.opendev.org/c/openstack/nova/+/922212 o:) | 14:03 |
| bauzas | elodilles: I'm just fighting with multinode zuul :) | 14:05 |
| elodilles | yeah i saw that o:) | 14:07 |
| bauzas | sean-k-mooney: I maybe need some help for debugging my mtty job https://59eb0550fb97fc327d84-51181049425e2f372c12ca87aad48047.ssl.cf2.rackcdn.com/922140/8/experimental/nova-mtty/8550c60/controller/logs/screen-br-ex-tcpdump.txt | 14:08 |
| bauzas | the multinode job fails ^ | 14:08 |
| bauzas | everytime you think you can understand OVN, then you find something you don't know | 14:12 |
| bauzas | just saying, the VTEP failed : https://59eb0550fb97fc327d84-51181049425e2f372c12ca87aad48047.ssl.cf2.rackcdn.com/922140/8/experimental/nova-mtty/8550c60/controller/logs/ovn/ovn-controller-vtep_log.txt | 14:14 |
| bauzas | maybe it's related if the public IPs are directly connected thru northbound OVN with br-ex | 14:15 |
| sean-k-mooney | well you can trun off the tcpdump service | 14:20 |
| sean-k-mooney | its just an extra dbuging thing for the ci | 14:20 |
| sean-k-mooney | and its really only used by the neuron team | 14:20 |
| sean-k-mooney | so for right now you could disabel it as a work around | 14:20 |
| bauzas | k | 14:21 |
| sean-k-mooney | ill quickly take a look at the devstack log and see if anything jumps out | 14:21 |
| sean-k-mooney | ya so looking at the ovs-vswitchd log on the compute https://59eb0550fb97fc327d84-51181049425e2f372c12ca87aad48047.ssl.cf2.rackcdn.com/922140/8/experimental/nova-mtty/8550c60/compute1/logs/openvswitch/ovs-vswitchd_log.txt | 14:23 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: WIP : Enable an experimental job for testing mtty https://review.opendev.org/c/openstack/nova/+/922140 | 14:23 |
| sean-k-mooney | there is not br-ex | 14:23 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: WIP : mtty: add devstack compilation for fedora builds https://review.opendev.org/c/openstack/nova/+/918420 | 14:23 |
| sean-k-mooney | which is normal | 14:23 |
| bauzas | I just disabled the service, we'll see | 14:24 |
| sean-k-mooney | so for ml2-ovs in DVR (distirbuted virtural router) mode we also create an instance of the l3 agent on all the compute and create a br-ex | 14:24 |
| sean-k-mooney | but for ovn we do not use the l3 agent and the br-ex is only created on the contoler | 14:24 |
| sean-k-mooney | so you basicaly shoudl not run that service on the subnode | 14:25 |
| sean-k-mooney | you can just disable it on both if you want however | 14:25 |
| sean-k-mooney | on the contoler the bride does exist https://59eb0550fb97fc327d84-51181049425e2f372c12ca87aad48047.ssl.cf2.rackcdn.com/922140/8/experimental/nova-mtty/8550c60/controller/logs/openvswitch/ovs-vswitchd_log.txt | 14:25 |
| sean-k-mooney | and tcpdump was installed | 14:26 |
| sean-k-mooney | bauzas: there is noting obvious in the journal or in the devstack log but my guess is eitehr there has been a command line change or something like apparmour is causing the comand to fail | 14:30 |
| bauzas | ack thanks for digging | 14:31 |
| sean-k-mooney | im not sure what 203/EXEC is evstack@br-ex-tcpdump.service: Main process exited, code=exited, status=203/EXEC | 14:31 |
| bauzas_ | yeah, that's a weird error code | 14:34 |
| bauzas_ | shit my bouncer restarted the conn | 14:34 |
| bauzas_ | shit my bouncer restarted the conn | 14:34 |
| *** bauzas_ is now known as bauzas | 14:39 | |
| *** dtantsur_ is now known as dtantsur | 15:12 | |
| bauzas | https://zuul.opendev.org/t/openstack/build/32d394d8265e4cc188fc3c5a50d33993 🎉 | 16:13 |
| bauzas | dang, Tempest is creating new flavors and they don't ask for VGPUs | 16:20 |
| bauzas | 2024-06-19 15:18:47.346 | ++ lib/tempest:configure_tempest:298 : openstack --os-cloud devstack-admin flavor create --id 42 --ram 128 --disk 1 --vcpus 1 --property hw_rng:allowed=True m1.nano | 16:21 |
| bauzas | sean-k-mooney: do you think I could change my own ansible script in https://review.opendev.org/c/openstack/nova/+/922140/9/playbooks/nova-mtty/pre.yaml to do post.yaml for updating all the flavors ? | 16:22 |
| bauzas | https://d2ba671e5a990e8f3d73-f7c3a0f9889b6932e080ae1513d0dd0b.ssl.cf1.rackcdn.com/922140/9/experimental/nova-mtty/32d394d/controller/logs/devstacklog.txt => here I see that my pre.yaml script updates all the pre-existing flavors with resources:VGPU=1 | 16:23 |
| bauzas | 2024-06-19 15:17:53.434 | ++ opt/stack/devstack/local.sh:main:8 : openstack flavor set --property resources:VGPU=1 1 | 16:23 |
| sean-k-mooney | you could but the flavors used by tempest by defautl have know ides | 16:23 |
| sean-k-mooney | namely 42 an 84 | 16:23 |
| sean-k-mooney | which are m1.nano and m1.micro | 16:23 |
| sean-k-mooney | so you dont need to update all of them you can just update the two used by tempest | 16:23 |
| sean-k-mooney | local.sh will be run after tempest has created all the flavors | 16:24 |
| bauzas | sean-k-mooney: if I use post.yaml tbc, right? | 16:26 |
| sean-k-mooney | no | 16:26 |
| sean-k-mooney | local.sh is used by devstack and included after tempest is configured but before tempest test are run | 16:26 |
| bauzas | because pre.yaml runs before tempest creates its own flavors | 16:26 |
| sean-k-mooney | pre runs before the run playbook | 16:26 |
| sean-k-mooney | so be fore devstack is run | 16:26 |
| sean-k-mooney | local.sh is run here https://github.com/openstack/devstack/blob/master/stack.sh#L1485-L1492 | 16:27 |
| bauzas | 2024-06-19 15:18:24.159 | ++ lib/tempest:install_tempest:815 : git_clone https://opendev.org/openstack/tempest.git /opt/stack/tempest master | 16:27 |
| bauzas | compared to 2024-06-19 15:17:50.794 | + ./stack.sh:main:1489 : [[ -x /opt/stack/devstack/local.sh ]] | 16:28 |
| bauzas | so pre-run is called before install_tempest | 16:28 |
| sean-k-mooney | pre-run is called before the run playbook | 16:28 |
| sean-k-mooney | so its run after all the parent jobs pre playbooks | 16:28 |
| sean-k-mooney | and be fore the current jobs run playbook | 16:29 |
| sean-k-mooney | so its before stack.sh is invoked | 16:29 |
| sean-k-mooney | the proable i think is your using the wrong path | 16:29 |
| sean-k-mooney | on sec | 16:29 |
| sean-k-mooney | your using /opt/stack/devstack/local.sh | 16:29 |
| bauzas | which is expected path, right? | 16:30 |
| sean-k-mooney | not in ci | 16:30 |
| sean-k-mooney | that is not where devstack will be cloned | 16:30 |
| bauzas | https://github.com/openstack/devstack/blob/master/stack.sh#L1489 is where my /opt/stack/devstack/local.sh script is called | 16:32 |
| bauzas | 2024-06-19 15:17:50.794 | + ./stack.sh:main:1489 : [[ -x /opt/stack/devstack/local.sh ]] | 16:32 |
| sean-k-mooney | oh actully i gues its correct | 16:32 |
| sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/679656/13/playbooks/nfv/nfv.yaml | 16:32 |
| sean-k-mooney | oh right | 16:32 |
| bauzas | but alas, install_tempest runs *after* stack.sh | 16:32 |
| sean-k-mooney | it should not | 16:32 |
| sean-k-mooney | we install tempet as part of devstack | 16:33 |
| bauzas | yeah I'm confused | 16:34 |
| sean-k-mooney | do you have a link to th job | 16:34 |
| bauzas | yeah | 16:34 |
| bauzas | https://zuul.opendev.org/t/openstack/build/32d394d8265e4cc188fc3c5a50d33993 | 16:34 |
| sean-k-mooney | im about to finish for today but ill take a look beofre i drop | 16:39 |
| bauzas | I think I found why | 16:40 |
| bauzas | 2024-06-19 15:18:23.933 | + ./stack.sh:main:1517 : run_phase stack test-config | 16:40 |
| bauzas | then runs 2024-06-19 15:18:23.963 | ++ functions-common:run_phase:1860 : basename /opt/stack/devstack/extras.d/80-tempest.sh | 16:40 |
| bauzas | which then does some async_wait 2024-06-19 15:18:24.047 | ++ extras.d/80-tempest.sh:source:20 : async_wait install_tempest | 16:41 |
| bauzas | which then installs tempest 2024-06-19 15:18:24.159 | ++ lib/tempest:install_tempest:815 : git_clone https://opendev.org/openstack/tempest.git /opt/stack/tempest master | 16:41 |
| sean-k-mooney | that sounds like a regression in devstack | 16:49 |
| bauzas | maybe I should create the flavors and then configure tempest to use them | 16:51 |
| sean-k-mooney | i have doen that in the past too | 16:52 |
| sean-k-mooney | it works | 16:52 |
| bauzas | do you have some job using it ? | 16:52 |
| sean-k-mooney | thats what i normally do https://review.opendev.org/c/openstack/nova/+/679656/13/playbooks/nfv/nfv.yaml | 16:52 |
| sean-k-mooney | bauzas: not that we merged | 16:52 |
| sean-k-mooney | but i created multi numa testing jobs a few times in the past | 16:53 |
| sean-k-mooney | just to show it workign | 16:53 |
| bauzas | cool | 16:53 |
| bauzas | I already knew how to use another flavor for Tempest https://docs.openstack.org/tempest/latest/configuration.html#flavors | 16:53 |
| bauzas | but I found how to modify the config by a CI job with your change : https://review.opendev.org/c/openstack/nova/+/679656/13/.zuul.yaml | 16:54 |
| bauzas | thanks | 16:54 |
| bauzas | I think it's better than just modifying all of the flavors | 16:54 |
| bauzas | anyway, I'm done too | 16:55 |
| bauzas | I'll change my stuff tomorrow morning | 16:55 |
| *** dasm is now known as Guest10177 | 17:13 | |
| opendevreview | Sylvain Bauza proposed openstack/nova master: WIP : Enable an experimental job for testing mtty https://review.opendev.org/c/openstack/nova/+/922140 | 19:10 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: WIP : mtty: add devstack compilation for fedora builds https://review.opendev.org/c/openstack/nova/+/918420 | 19:10 |
| *** iurygregory_ is now known as iurygregory | 21:28 | |
| *** bauzas_ is now known as bauzas | 22:39 | |
| sean-k-mooney[m] | bauzas: frickler just an fyi i was able to stack on nobe fine by just installing the latest version of setuptool in the requiments and global venv | 22:43 |
| sean-k-mooney[m] | if i get a chance tomorrow ill see if i can tweak devstack to make that work out of the box but i dont think we need to do anything speciall beyond that | 22:45 |
| opendevreview | sean mooney proposed openstack/nova master: WIP : Enable an experimental job for testing mtty https://review.opendev.org/c/openstack/nova/+/922140 | 22:47 |
| opendevreview | sean mooney proposed openstack/nova master: Revert "Always delete NVRAM files when deleting instances" https://review.opendev.org/c/openstack/nova/+/922354 | 22:51 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!