| opendevreview | Ghanshyam proposed openstack/nova stable/train: DNM: test tempest train-last tag https://review.opendev.org/c/openstack/nova/+/816598 | 00:47 |
|---|---|---|
| *** ministry is now known as __ministry | 02:26 | |
| *** gibi_pto_back_thu is now known as gibi | 07:56 | |
| gibi | good morning | 08:01 |
| opendevreview | Balazs Gibizer proposed openstack/nova stable/victoria: Parse alias from domain hostdev https://review.opendev.org/c/openstack/nova/+/816486 | 08:03 |
| opendevreview | Balazs Gibizer proposed openstack/nova stable/xena: Prevent leaked eventlets to send notifications https://review.opendev.org/c/openstack/nova/+/816487 | 08:07 |
| opendevreview | Balazs Gibizer proposed openstack/nova stable/xena: Avoid unbound instance_uuid var during delete https://review.opendev.org/c/openstack/nova/+/816488 | 08:08 |
| gibi | bauzas: the reno for the pps work has landed so you can marke the | 08:09 |
| gibi | https://blueprints.launchpad.net/nova/+spec/qos-minimum-guaranteed-packet-rate | 08:09 |
| gibi | bp as implemented | 08:09 |
| gibi | \o/ | 08:09 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Enable min pps tempest testing in nova-next https://review.opendev.org/c/openstack/nova/+/811748 | 08:11 |
| lyarwood | gibi++ | 08:15 |
| gibi | lyarwood: thanks for the reviews yestarday | 08:16 |
| lyarwood | np, still catching up | 08:16 |
| gibi | I hope you enjoyed the explanation of the fix for https://launchpad.net/bugs/1946339 . It was a real adventure for me in eventlet/greenlet land. | 08:20 |
| jpodivin | Hi everyone. Has anyone ever encountered: "The placement service for 192.168.24.100:regionOne exists but does not have any supported versions." ? | 08:21 |
| jpodivin | It's hitting master in rdo, but wallaby somehow avoids it. | 08:22 |
| gibi | jpodivin: hi, could you try to simply send a GET request to the root of you placement service? | 08:26 |
| gibi | you should see something like https://paste.opendev.org/show/810375/ | 08:26 |
| jpodivin | gibi: hm, sadly it's a proposed CI job, so the nodes are already down. | 08:27 |
| jpodivin | althought I do have the logs stored | 08:27 |
| gibi | then it would be good to check the placement service logs | 08:27 |
| jpodivin | gibi: anything specific I should look for? | 08:28 |
| gibi | jpodivin: do you see requests resulting in 404 there? | 08:28 |
| EugenMayer | when running nova backup on an image, i see pending operations in the UI. Is there any way to show this tasks and their detailed operations on the cli? glance taks-list and glance image-tasks do not show any pending tasks | 08:28 |
| gibi | we had an issue with apache config not long ago resulted in wrong proxy config for placement https://review.opendev.org/c/openstack/devstack/+/811389 | 08:29 |
| jpodivin | gibi: no, it looks fairly clean, couple of warnings but those seem related to deprecation. https://logserver.rdoproject.org/99/36499/6/check/periodic-tripleo-ci-centos-8-containers-multinode-compute-master-validation/91262ed/logs/subnode-1/var/log/containers/placement/placement.log.txt.gz | 08:29 |
| jpodivin | gibi: found another log, that might be what you meant. Lists a bunch of requests but 404 https://logserver.rdoproject.org/99/36499/6/check/periodic-tripleo-ci-centos-8-containers-multinode-compute-master-validation/91262ed/logs/subnode-1/var/log/containers/httpd/placement/placement_wsgi_access.log.txt.gz | 08:31 |
| jpodivin | Interesting thing is: it starts *after* the error occurs, not before. | 08:32 |
| gibi | yeah, these logs seems to indicate your placement service works correctly and some clients (probably nova) was able to communicate with it | 08:32 |
| gibi | if the error happens after the logs ends then it can be that your placement service was simply stopped / crashed | 08:33 |
| jpodivin | gibi: looking at the time stamps, it looks like the error occurs a approximately one minute before the placement requests log starts . | 08:34 |
| jpodivin | gibi: scrap that. It's actually about 3 seconds before | 08:34 |
| jpodivin | still, before any communication is recorded . | 08:35 |
| gibi | still it means that the placement server is not running when you client tried to read the supported versions from it | 08:35 |
| lyarwood | bauzas / melwitt ; https://review.opendev.org/c/openstack/nova/+/807025 - would you mind hitting this and the fups on top this week? | 08:40 |
| jpodivin | gibi: yes that does seem to be the case | 08:45 |
| jpodivin | gibi: question is: why would that happen on master and not on wallaby. | 08:45 |
| gibi | jpodivin: I have no idea | 09:06 |
| lyarwood | sean-k-mooney: can you hit https://review.opendev.org/c/openstack/nova/+/811716 again when you get a chance | 09:34 |
| EugenMayer | after deploying with kolla and tls, it seems like the GUI based TTY console is still behin to http, not https, while anything else is properly behind tls encryption. The point is, horizon tries to load from https:6080 .. but it fails (no socket). http:6080 has a socket, but the token seems to be invalid there. Any hints what could be wrong, i | 09:35 |
| EugenMayer | assume this is related to the nova service? | 09:35 |
| opendevreview | Lee Yarwood proposed openstack/nova master: WIP configdrive: Move mkisofs_cmd default to mkisofs https://review.opendev.org/c/openstack/nova/+/808921 | 09:36 |
| EugenMayer | when looking at the nova-compute configuration i see https://gist.github.com/EugenMayer/058499029fbd298600a8efa634687c92 | 09:36 |
| bauzas | gibi: done, https://blueprints.launchpad.net/nova/+spec/qos-minimum-guaranteed-packet-rate is now Implemented | 09:37 |
| gibi | bauzas: thanks | 09:37 |
| bauzas | (thanks for noticing me) | 09:38 |
| bauzas | lyarwood: ack, will look | 09:38 |
| EugenMayer | soseems like https://bugzilla.redhat.com/show_bug.cgi?id=1722089 is related | 09:38 |
| opendevreview | Lee Yarwood proposed openstack/nova master: DNM/WIP libvirt: Default x86_64 instances to the q35 machine type https://review.opendev.org/c/openstack/nova/+/816629 | 09:40 |
| kashyap | lyarwood: --^ I'm curious too... :) Thx for posting it | 09:41 |
| lyarwood | EugenMayer: the downstream Red hat bug isn't related | 09:41 |
| EugenMayer | lyarwood thank you. Not sure what i miss then, reading https://docs.openstack.org/nova/xena/admin/remote-console-access.html | 09:42 |
| EugenMayer | a little hard to understand what kolla did and did not | 09:42 |
| EugenMayer | this is what i have in nova-vnc https://gist.github.com/EugenMayer/82528fcfca6e22b818f865852606f28c - currently kind of double posting since i'am not sure this is a nova 'issue' or a kolla 'configuration' issue or a kolla 'deployment bug' | 09:45 |
| lyarwood | EugenMayer: I'd think that's a kolla config bug tbh | 09:47 |
| lyarwood | the URL returned from n-api is https right? | 09:47 |
| lyarwood | when you do a console url show? | 09:47 |
| EugenMayer | lyarwood how would i do that? | 09:48 |
| lyarwood | EugenMayer: https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/console-url.html#console-url-show openstack console url show $instance | 09:51 |
| EugenMayer | lyarwood the URL looks good. HTTPS, right fqdn, 60800 | 09:54 |
| EugenMayer | telent on that port works, but i assume it talks http | 09:55 |
| EugenMayer | wget returns 'GnuTLS: Error in the pull function. | 09:56 |
| EugenMayer | Unable to establish SSL connection.' | 09:56 |
| lyarwood | kk then it's the nova-novncproxy service that isn't configured correctly and thus a Kolla bug | 09:56 |
| EugenMayer | lyarwood thank you. Already looking at the configuration. I assume the certificates are not bound properly | 09:59 |
| EugenMayer | lyarwood https://gist.github.com/EugenMayer/fba4eb20a49ccd717ba70f38188a8e1e - i'am using officially signed certificates. Maybe missing https://docs.openstack.org/nova/xena/admin/remote-console-access.html#novnc-proxy-server-configuration /etc/pki/libvirt-vnc/server-cert.pem or something? | 10:02 |
| lyarwood | that's to encrypt the connection between the proxy service and libvirtd | 10:04 |
| lyarwood | I think your issue is between the user and the proxy service right? | 10:05 |
| EugenMayer | yes | 10:10 |
| EugenMayer | lyarwood any hints where to start | 10:29 |
| lyarwood | EugenMayer: I'm not sure how Kolla configures things tbh but I'd start with the config file associated with the nova-novncproxy service itself | 10:29 |
| lyarwood | EugenMayer: see if it differs from nova-api etc that are working with tls already | 10:30 |
| EugenMayer | that's what i posted lyarwood - the problem is, not sure what to expect there | 10:30 |
| *** lbragstad4 is now known as lbragstad | 10:33 | |
| opendevreview | Merged openstack/nova stable/xena: Store old_flavor already on source host during resize https://review.opendev.org/c/openstack/nova/+/810911 | 10:34 |
| lyarwood | EugenMayer: https://github.com/openstack/nova/blob/909cfc76369b94b026cf42b86fb5a310dce21a8c/nova/cmd/baseproxy.py#L48-L87 - so looking at the code it's using configurables like ssl_only and cert | 10:36 |
| EugenMayer | which are missing in my case | 10:37 |
| lyarwood | EugenMayer: iirc from an earlier pastebin websockify couldn't find the cert? | 10:37 |
| lyarwood | I've lost the gist now but I'm sure it was listing a cert file | 10:39 |
| lyarwood | so you must have cert set to something in the config used by the proxy service | 10:39 |
| lyarwood | it's in the DEFAULT namespace btw, not under vnc | 10:39 |
| lyarwood | just grep from ^cert | 10:39 |
| lyarwood | for* | 10:40 |
| *** lbragstad1 is now known as lbragstad | 10:43 | |
| EugenMayer | sorry i'am lost lyarwood, not sure which you mean https://gist.github.com/EugenMayer/058499029fbd298600a8efa634687c92 or https://gist.github.com/EugenMayer/fba4eb20a49ccd717ba70f38188a8e1e or https://gist.github.com/EugenMayer/82528fcfca6e22b818f865852606f28c - if nothing of this, which on do you need. Happy to hand over anything | 10:45 |
| lyarwood | 2021-11-04 09:18:39.933 13300 INFO nova.console.websocketproxy [-] 10.0.1.1: SSL connection but '/self.pem' not found | 10:45 |
| lyarwood | ^ that error suggests that the nova-novncproxy hasn't been configured correctly | 10:46 |
| EugenMayer | yes, that is what i saw too. So 'cert' in noproxy under [vnc] | 10:50 |
| opendevreview | Wenping Song proposed openstack/nova master: Support concurrently add hosts to aggregates https://review.opendev.org/c/openstack/nova/+/815105 | 10:50 |
| lyarwood | EugenMayer: no, it's outside of that in the default namespace AFAICT | 10:51 |
| lyarwood | EugenMayer: if you grep for self.pem you should be able to find it | 10:51 |
| lyarwood | EugenMayer: this is definitely a Kolla bug FWIW | 10:51 |
| lyarwood | EugenMayer: it smells like it hasn't copied that cert into the container for the service or something? | 10:52 |
| EugenMayer | it does copy those and maybe is missing one. Let me clear up my confusion Which part of the configuration is broken / missing ther certs. On the computes the nova-compute or on the controller the novncproxy | 10:53 |
| lyarwood | EugenMayer: on the contrller, the novncproxy service | 10:55 |
| lyarwood | EugenMayer: assuming the logs you shared were from the controller | 10:57 |
| lyarwood | EugenMayer: regarding 2021-11-04 09:18:39.933 13300 INFO nova.console.websocketproxy [-] 10.0.1.1: SSL connection but '/self.pem' not found | 10:57 |
| lyarwood | Yeah actually that can only be from the controller | 10:58 |
| EugenMayer | lyarwood ok so i check the noproxy container and how the certificates are deployed and configured | 11:02 |
| lyarwood | https://github.com/novnc/websockify/blob/master/README.md#encrypted-websocket-connections-wss - FWIW self.pem is the default cert websockify will try to load when it's asked to use SSL | 11:05 |
| lyarwood | as I said before https://github.com/openstack/nova/blob/909cfc76369b94b026cf42b86fb5a310dce21a8c/nova/cmd/baseproxy.py#L48-L87 is where Nova tries to provide the correct cert when launching the novncproxy service | 11:06 |
| EugenMayer | well ok now i guess that will be the issue. is self.pem a cert+private-key format? | 11:07 |
| *** lbragstad7 is now known as lbragstad | 11:07 | |
| EugenMayer | ah --cert --key | 11:08 |
| lyarwood | right you should already have these in your env? | 11:10 |
| lyarwood | just under a different filename | 11:10 |
| lyarwood | so just update the nova.conf used by the service to point to them | 11:10 |
| lyarwood | cert=/path/to/cert | 11:10 |
| lyarwood | key=/path/to/key | 11:10 |
| lyarwood | and again, DEFAULT namespace so outside of the [vnc] section etc. | 11:11 |
| EugenMayer | checking the ansible tasks right now (kollas) | 11:11 |
| lyarwood | yeah it doesn't look like it has support tbh | 11:16 |
| EugenMayer | https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/nova-cell/templates/nova.conf.j2 | 11:16 |
| lyarwood | looking at the config templates at least | 11:16 |
| EugenMayer | yes, it's mising | 11:16 |
| EugenMayer | lyarwood did the nova implemenation of novnc change since victoria? | 11:17 |
| lyarwood | I don't think anything has that would change this behaviour tbh | 11:18 |
| EugenMayer | in other words, looking back, that template never had cert/key as values set for TLS | 11:18 |
| EugenMayer | so either it has never been supported at all - or it is a regression | 11:19 |
| lyarwood | yeah I would assume this has never been supported by Kolla tbh, should be pretty trivial to correct however | 11:20 |
| EugenMayer | yes it is just PITA to search for that, i you are clueless (like i'am). You never know what is supposed to work and what not, and how a working configuration does look like | 11:21 |
| EugenMayer | lyarwood any idea what the path of self.pem looks like? i mean i do not assume /self.pem is really absolute here | 11:22 |
| lyarwood | https://github.com/openstack/nova/blob/909cfc76369b94b026cf42b86fb5a310dce21a8c/nova/conf/novnc.py#L41-L52 looks like it's relative so it depends how kolla is launching the service | 11:24 |
| lyarwood | but again updating the nova.conf used by the service to point to your actual key and cert is a better option here | 11:24 |
| EugenMayer | yes sure, it is a little more work then that | 11:25 |
| EugenMayer | i will need to volume-mount the certs first, i cannot just docker cp them, or they will be lost on upgrade. Then a config override for nova conf (conditional) and then mounting that certs | 11:26 |
| EugenMayer | but still, HUGE help lyarwood, i should be able to handle the rest. Thank you big times! | 11:26 |
| lyarwood | np good luck :) | 11:26 |
| EugenMayer | thank you sir | 11:26 |
| *** dpawlik1 is now known as dpawlik | 12:16 | |
| opendevreview | Vlad Gusev proposed openstack/nova stable/train: Reproduce bug 1897528 https://review.opendev.org/c/openstack/nova/+/792116 | 12:24 |
| opendevreview | Vlad Gusev proposed openstack/nova stable/train: Ignore PCI devices with 32bit domain https://review.opendev.org/c/openstack/nova/+/792117 | 12:24 |
| opendevreview | Vlad Gusev proposed openstack/nova stable/stein: Reproduce bug 1897528 https://review.opendev.org/c/openstack/nova/+/816656 | 12:25 |
| opendevreview | Vlad Gusev proposed openstack/nova stable/stein: Reproduce bug 1897528 https://review.opendev.org/c/openstack/nova/+/816656 | 12:26 |
| opendevreview | Vlad Gusev proposed openstack/nova stable/stein: Reproduce bug 1897528 https://review.opendev.org/c/openstack/nova/+/816656 | 12:37 |
| opendevreview | Vlad Gusev proposed openstack/nova stable/stein: Ignore PCI devices with 32bit domain https://review.opendev.org/c/openstack/nova/+/816682 | 12:37 |
| opendevreview | Vlad Gusev proposed openstack/nova stable/stein: Ignore PCI devices with 32bit domain https://review.opendev.org/c/openstack/nova/+/816682 | 13:20 |
| opendevreview | Merged openstack/nova master: compute: Update volume_id within connection_info during swap_volume https://review.opendev.org/c/openstack/nova/+/807025 | 13:24 |
| opendevreview | Merged openstack/nova master: fup: Move _wait_for_volume_{attach,detach} to os-volume_attachments https://review.opendev.org/c/openstack/nova/+/810775 | 13:24 |
| opendevreview | Merged openstack/nova master: fup: Refactor and simplify Cinder fixture GET volume mock https://review.opendev.org/c/openstack/nova/+/810776 | 13:25 |
| opendevreview | Merged openstack/nova master: Clean up allocations left by evacuation when deleting service https://review.opendev.org/c/openstack/nova/+/778696 | 14:20 |
| opendevreview | Merged openstack/nova stable/wallaby: Reproduce bug 1944759 https://review.opendev.org/c/openstack/nova/+/810912 | 14:20 |
| gibi | lyarwood: hi! it seems there is a variant of https://bugs.launchpad.net/nova/+bug/1931702 in https://zuul.opendev.org/t/openstack/build/582935ad35a348cf89dcb25bdc3be0ea/logs But the guest console log at volume detach is different now https://zuul.opendev.org/t/openstack/build/582935ad35a348cf89dcb25bdc3be0ea/log/controller/logs/tempest_log.txt#5444 | 14:53 |
| gibi | elodilles: ^^ | 14:53 |
| gibi | "[ 15.981709] virtio_blk virtio4: req.0:id 4 is not a head!" | 14:53 |
| gibi | lyarwood: does it ring a bell for you? | 14:54 |
| lyarwood | gibi: no I've not seen that before tbh | 14:56 |
| gibi | lyarwood: ack, thanks | 14:56 |
| artom_ | bauzas, hey, I think the Ironic folks would be really happy if we made https://review.opendev.org/c/openstack/nova/+/813263 a review priority... | 15:02 |
| *** artom_ is now known as artom | 15:02 | |
| sean-k-mooney | i see | 15:03 |
| sean-k-mooney | i think should be safe although it raise the question about oter life cyle events liek power on power off and had/soft reboot | 15:04 |
| artom | sean-k-mooney, the Ironic patch? | 15:05 |
| sean-k-mooney | yes | 15:05 |
| artom | Yeah, I suppose it does, but from what I've seen, use of plug_vifs() is highly limited, so it's safe to make it a noop | 15:05 |
| sean-k-mooney | no its not | 15:05 |
| sean-k-mooney | we need to call it for the inial spawn | 15:05 |
| artom | sean-k-mooney, I mean, look at my review notes inline, and tell me if I've missed something :) | 15:06 |
| sean-k-mooney | we do not need to call it in init_host for ironci | 15:06 |
| sean-k-mooney | but it cant jsut be a noop without change the spwan workflow | 15:06 |
| sean-k-mooney | we use it on inital boot to ensure that the networkign if fully configured by the backend before we power on the ironic host | 15:07 |
| artom | sean-k-mooney, maybe you're thinking of a slightly differently named method? | 15:07 |
| sean-k-mooney | no im not | 15:07 |
| artom | In the compute manager, it's only called from _init_instance(), which is only called from init_host() | 15:08 |
| artom | That's it, nothing on spawn | 15:08 |
| sean-k-mooney | correct its not | 15:08 |
| sean-k-mooney | but we also call plug_vifs during spwan | 15:08 |
| sean-k-mooney | so you cant just make plug_vifs a noop | 15:08 |
| artom | From where? | 15:09 |
| sean-k-mooney | it will mean during spawn we will not actully set up the networking proerly they have hacked around this here https://review.opendev.org/c/openstack/nova/+/813263/3/nova/virt/ironic/driver.py#1606 | 15:09 |
| sean-k-mooney | by starting to use _plug_vifs to actully invoke the ironic api | 15:09 |
| sean-k-mooney | for interface attach | 15:09 |
| artom | That's just inlining what plug_vifs() used to do into attach(), no? | 15:10 |
| bauzas | artom: I can mark it as a Review-Priority for me | 15:10 |
| * bauzas jumped into the unified limits series | 15:11 | |
| artom | bauzas, PTL's discretion and all that :) I was just making a request | 15:11 |
| bauzas | any core can set this flag | 15:12 |
| bauzas | ... for the moment | 15:12 |
| artom | But you're the core-iest of cores | 15:12 |
| bauzas | I'm about to write a doc change for it, hopefully tomorow | 15:12 |
| bauzas | artom: nah, as sean-k-mooney said, I'm just a "cat herder" or if you prefer, some French guy yelling in the wind | 15:12 |
| gibi | sean-k-mooney, artom: I read that ironic related nova patch, I see that it is correct and does not affect spawn, but now I'm affraid that sean-k-mooney has things I'm missing | 15:13 |
| artom | Meow. | 15:13 |
| artom | gibi, you and me both | 15:13 |
| gibi | we need more cats | 15:13 |
| bauzas | I have a dog | 15:13 |
| gibi | then you are out | 15:13 |
| gibi | :P | 15:13 |
| bauzas | cats are selfish | 15:13 |
| artom | Ionesco says dogs are cats | 15:13 |
| sean-k-mooney | gibi: im mostly uncofrotable with changing the meaing of plug_vifs to be honest | 15:13 |
| artom | He also says that Socrates was a cat | 15:13 |
| sean-k-mooney | im currently reviewing the spawn path | 15:14 |
| artom | sean-k-mooney, I don't think that's our call to make, every driver can do what they want | 15:14 |
| bauzas | artom: the only merit to cats is that they help to prove some theorem | 15:14 |
| bauzas | about quantic nature | 15:14 |
| artom | Only if they're in boxes | 15:15 |
| sean-k-mooney | artom: its used here https://github.com/openstack/nova/blob/fded762f4df26ff5706438a66da33ff966f833c6/nova/virt/ironic/driver.py#L1910 | 15:16 |
| sean-k-mooney | which is used by the compute manger here https://github.com/openstack/nova/blob/fded762f4df26ff5706438a66da33ff966f833c6/nova/compute/manager.py#L2597 | 15:17 |
| sean-k-mooney | in _build_resources | 15:17 |
| gibi | sean-k-mooney: our virt driver interface has the plug_vif method but we only use that from the computa manager in init_host | 15:17 |
| sean-k-mooney | which is part of _build_and_run_instance | 15:17 |
| gibi | sean-k-mooney: that call is transformed out in the proposed patch | 15:17 |
| artom | sean-k-mooney, yeah, and that's been inlined here: https://review.opendev.org/c/openstack/nova/+/813263/3/nova/virt/ironic/driver.py#1919 | 15:17 |
| gibi | sean-k-mooney: and that call paths is totaly ironic specific, libvirt virt driver does not call back to plug_vifs during spawn | 15:18 |
| artom | AFAICT, TheJulia did her homework :) | 15:18 |
| sean-k-mooney | oh that prepare_networks_before_block_device_mapping | 15:18 |
| * TheJulia appears having been summoned | 15:18 | |
| sean-k-mooney | gibi: well we do use it internally in the virt dirver i think in livemigration and maybe hard reboot | 15:19 |
| sean-k-mooney | but perhaps not directly in spwan | 15:19 |
| gibi | sean-k-mooney: not the virt driver interface | 15:19 |
| sean-k-mooney | gibi not the virt dirver interface not but the funciton in the dirver | 15:19 |
| artom | TheJulia, we're trying to convince sean-k-mooney you're plug_vifs() patch is correct :) | 15:19 |
| artom | *your | 15:19 |
| TheJulia | oh, yes, it is correct as far as I've been able to navigate | 15:19 |
| artom | Oh god, I'm one of them now | 15:19 |
| TheJulia | since ironic handles all attachment management | 15:19 |
| artom | The people who put apostrophe's in plural's and no apostrophe's in possessive's | 15:20 |
| sean-k-mooney | TheJulia: it was the swan path i was concerned about i had not expanded the context lins areound https://review.opendev.org/c/openstack/nova/+/813263/3/nova/virt/ironic/driver.py#1919 | 15:20 |
| sean-k-mooney | TheJulia: the change you made in prepare_networks_before_block_device_mapping will ensure we actully plug the vifs still on spwan | 15:20 |
| TheJulia | and due to that, and the duality use of attach_interfaces being present, it is entirely redundant and harmful behavior for the operators to experience with nova-computes running the ironic driver | 15:21 |
| gibi | sean-k-mooney: I see | 15:21 |
| artom | TheJulia, what does plugging the vifs mean in an Ironic context, anyways? Calling out to a physical switch and setting vlans and stuff? | 15:21 |
| sean-k-mooney | TheJulia: what i was really concerned about is regressing spawn such that the compute manger would not wait for the vifs to be configured on the network swtich berofre powering on the server | 15:22 |
| sean-k-mooney | but it looks like that cant happen so ill +1 it shortly | 15:22 |
| TheJulia | artom: recording there should be an attachment, that is only sent to neutron until once the states change to active | 15:22 |
| TheJulia | sean-k-mooney: those are persistant and managed through the workflow and neutron ml2 plugins | 15:22 |
| TheJulia | sean-k-mooney: but totally valid concern not knowing the rest of the mechanics | 15:23 |
| sean-k-mooney | TheJulia: well i mean this is not something that ironic should really mange entrily on its own so jsut making sure we still have the correct mechamins in place on the nova side | 15:24 |
| sean-k-mooney | altough looking at that code path we dont seam to be correctly waiting for the neutron external event in _plug_vifs | 15:25 |
| sean-k-mooney | https://github.com/openstack/nova/blob/fded762f4df26ff5706438a66da33ff966f833c6/nova/virt/ironic/driver.py#L1492-L1525 | 15:25 |
| sean-k-mooney | we are just callign the node.vif_attach api | 15:26 |
| TheJulia | sean-k-mooney: it must because of a security lifecycle must be enforced | 15:26 |
| sean-k-mooney | so this looks like there is an existing race | 15:26 |
| TheJulia | and it knows the state of the lifecycle | 15:26 |
| EugenMayer | Anybody in here got novnc working with kolla when using TLS? TLS is working on all sub-systems except when using TLS. lyarwood it seems like they use a haproxy (i got told) which does the SSL offloading, which might be the reason it is not configured. But this would not explain the error message | 15:27 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Add a WA flag waiting for vif-plugged event during reboot https://review.opendev.org/c/openstack/nova/+/813419 | 15:27 |
| TheJulia | sean-k-mooney: on start, but if the vif recrods are already on file, on start it doesn't matter, the nova-compute spins for quite a long time. | 15:29 |
| TheJulia | for running state, that is a separate path and that has been the case for a while, I think. | 15:30 |
| TheJulia | so I think I grok sean-k-mooney's concerns, and just to be on the safe side, I'll go change my "test nova-y tings patch in the ironic repo to pull that vif plug patch in and just make sure that it passes happily again. I believe it did so before, but there is the BFV use case which is a little different | 15:50 |
| opendevreview | Alexey Stupnikov proposed openstack/nova master: WIP: Test aborting queued live migration https://review.opendev.org/c/openstack/nova/+/776250 | 15:54 |
| TheJulia | artom: that hash ring handling wip seems to work, which is a good sign. Likely just need to go see if the devstack plugin forces a rebalance... and maybe make it force a rebalance :) | 15:55 |
| artom | TheJulia, yeah, I need to look at it again and properly wrap my head around it | 15:55 |
| TheJulia | artom: yeah, a little different by just reconciling "what is running/active" versuse explicit record checks for each instance, but I'd hate to trigger a few thousand extra DB queries upon rebalance | 15:57 |
| TheJulia | changed https://review.opendev.org/c/openstack/ironic/+/813264 to run the vif plug change | 15:59 |
| TheJulia | sean-k-mooney: ^^ if the ironic bfv job passes, I suspect we're safe and happy for the time being | 15:59 |
| sean-k-mooney | ack tahnks | 16:03 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Remove SESSION_CONFIGURED global from DB fixture https://review.opendev.org/c/openstack/nova/+/815689 | 16:27 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Refactor Database fixture https://review.opendev.org/c/openstack/nova/+/815690 | 16:27 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Fix interference in db unit test https://review.opendev.org/c/openstack/nova/+/814735 | 16:29 |
| melwitt | gibi: just read your comments, I might be wrong but if I am then I don't understand why it's needed. I'll look at it some more | 16:32 |
| gibi | melwitt: I followed your suggestion and things are still passing, so I think you are right | 16:33 |
| melwitt | oh ok | 16:33 |
| gibi | so you were right that I don't need to patch the per connection case | 16:34 |
| melwitt | ack | 16:34 |
| EugenMayer | lyarwood FYI - kolla supports tls for novnc but uses a haproxy as SSL offloaded and LB in front of it. My issues was an internal vs external VIP IP. So kolla has support for it | 18:00 |
| lyarwood | cool cool | 18:01 |
| opendevreview | Lee Yarwood proposed openstack/nova master: nova-next: Deploy noVNC from source instead of packages https://review.opendev.org/c/openstack/nova/+/816738 | 18:34 |
| opendevreview | Lee Yarwood proposed openstack/nova master: nova-next: Drop NOVA_USE_SERVICE_TOKEN from subnode https://review.opendev.org/c/openstack/nova/+/816740 | 18:41 |
| opendevreview | Gustavo Santos proposed openstack/nova master: Reattach mdevs to guest on resume https://review.opendev.org/c/openstack/nova/+/815373 | 20:31 |
| opendevreview | sean mooney proposed openstack/nova master: [WIP] Add extra tests for pinning with partial siblings https://review.opendev.org/c/openstack/nova/+/816758 | 22:07 |
| opendevreview | Merged openstack/nova stable/wallaby: Store old_flavor already on source host during resize https://review.opendev.org/c/openstack/nova/+/810913 | 23:30 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!