| opendevreview | Dan Smith proposed openstack/nova master: Test ceph-multistore with a real image https://review.opendev.org/c/openstack/nova/+/860864 | 00:56 |
|---|---|---|
| opendevreview | Merged openstack/nova stable/yoga: Adds a repoducer for post live migration fail https://review.opendev.org/c/openstack/nova/+/861871 | 01:34 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/ussuri: add regression test case for bug 1978983 https://review.opendev.org/c/openstack/nova/+/862603 | 05:58 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/ussuri: For evacuation, ignore if task_state is not None https://review.opendev.org/c/openstack/nova/+/862604 | 05:58 |
| ygk_12345 | hi all | 08:59 |
| ygk_12345 | I am trying to use pdb trace in the nova-api service in OSA setup. but when I start it manually , it is not dropping into the pdb shell. can anyone help me here ? | 09:01 |
| bauzas | ygk_12345: you need to run a WSGI server that can support pdb | 09:01 |
| bauzas | ygk_12345: example with httpd mod_wsgi https://modwsgi.readthedocs.io/en/master/user-guides/debugging-techniques.html#python-interactive-debugger | 09:04 |
| bauzas | simple way with uwsgi https://stackoverflow.com/questions/18427948/how-to-debug-python-application-under-uwsgi | 09:06 |
| ygk_12345 | bauzas: doesn't OSA nova-api support pdb ? | 09:14 |
| bauzas | ygk_12345: I suppose nova-api wsgi app runs on a wsgi server (which I guess is httpd) which runs on daemon mode | 09:15 |
| bauzas | if it would be uwsgi, that would still be a daemon | 09:16 |
| bauzas | if you wanna use pdb, you need to run the wsgi server on demand and not as a daemon thru systemd or whatever | 09:16 |
| bauzas | or rather use a remote pdb | 09:17 |
| bauzas | but then you would need to make sure that you run on a single nova-api worker | 09:18 |
| ygk_12345 | bauzas: I am using this manual command "/openstack/venvs/uwsgi-20.2.1-python3/bin/uwsgi --ini /etc/uwsgi/nova-api-os-compute.ini " | 09:19 |
| ygk_12345 | bauzas: I have inserted importpdb;pdb.set_trace() in the file "/openstack/venvs/nova-20.2.1/lib/python3.6/site-packages/nova/objects/instance.py . but I see no pdb prompt | 09:20 |
| bauzas | ygk_12345: you should probably add --honour-stdin and --workers 1 to the uwsgi command | 09:23 |
| ygk_12345 | bauzas: let me try that | 09:23 |
| bauzas | ygk_12345: and then, add a breakpoint in some easy rest endpoint call, like in nova.openstack.compute.api.servers on the get_instances() method | 09:24 |
| bauzas | oh wait, you're not getting a BdBQuit error ? | 09:25 |
| bauzas | that means that you haven't walked yet to where you added the breakpoint | 09:25 |
| ygk_12345 | bauzas: still no luck | 09:26 |
| ygk_12345 | bauzas: i am running the command nova list | 09:26 |
| bauzas | ygk_12345: yup, because as i said just above, you haven't went to that line of code | 09:26 |
| bauzas | s/went/gone | 09:26 |
| ygk_12345 | get_all functions is executed from /openstack/venvs/nova-20.2.1/lib/python3.6/site-packages/nova/objects/instance.py right ? | 09:27 |
| bauzas | ygk_12345: oh, you're not looking at the nova-api logs ? | 09:27 |
| ygk_12345 | bauzas: no, I am tracing the code | 09:27 |
| bauzas | ygk_12345: just to clarify, you won't get a prompt on your client call | 09:28 |
| ygk_12345 | bauzas: yes | 09:28 |
| bauzas | the interactive debugging session will run on the nova-api process that executes the server-side code for listing the instances | 09:28 |
| bauzas | you can trigger it thru a client call of course | 09:29 |
| bauzas | ygk_12345: to answer your question, no, a nova list doesn't lookup the instance records | 09:31 |
| bauzas | ahah, I see your problem | 09:31 |
| bauzas | since Newton, we don't expose the Instance record when you nova list | 09:32 |
| bauzas | we rather expose an InstanceMapping record, eventually a BuildRequest object | 09:32 |
| bauzas | you should rather try a nova show | 09:32 |
| ygk_12345 | can anyone help me with this ? | 09:36 |
| opendevreview | Rajesh Tailor proposed openstack/os-vif stable/zed: Move mtu update request into ovsdb transaction https://review.opendev.org/c/openstack/os-vif/+/863993 | 11:31 |
| opendevreview | Rajesh Tailor proposed openstack/os-vif stable/yoga: Move mtu update request into ovsdb transaction https://review.opendev.org/c/openstack/os-vif/+/863994 | 11:32 |
| opendevreview | Rajesh Tailor proposed openstack/os-vif stable/xena: Move mtu update request into ovsdb transaction https://review.opendev.org/c/openstack/os-vif/+/863995 | 11:34 |
| admin1 | hi all .. my nova-api logs is filled with ERROR oslo_service.periodic_task oslo_ db.exception.DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query' .. but it reconnects .. and this is happening every 1-2 mins .. other apis are fine | 12:15 |
| admin1 | am I hitting some kind of nova limits or connection/cache timeouts | 12:15 |
| admin1 | mysql connection limit is not reached | 12:15 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/ussuri: Adds a repoducer for post live migration fail https://review.opendev.org/c/openstack/nova/+/864006 | 13:57 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/ussuri: [compute] always set instance.host in post_livemigration https://review.opendev.org/c/openstack/nova/+/864007 | 13:57 |
| ygk_12345 | bauzas: now I am able to get the pdb prompt in the uwsgi process, but whatever pdb commands I type are not executing. also the cursor is coming out of pdb prompt by one line and waiting there | 14:01 |
| *** dasm|off is now known as dasm | 14:02 | |
| opendevreview | Dan Smith proposed openstack/nova master: Test ceph-multistore with a real image https://review.opendev.org/c/openstack/nova/+/860864 | 14:36 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/ussuri: Adds a repoducer for post live migration fail https://review.opendev.org/c/openstack/nova/+/864006 | 14:47 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/ussuri: [compute] always set instance.host in post_livemigration https://review.opendev.org/c/openstack/nova/+/864007 | 14:47 |
| opendevreview | Dan Smith proposed openstack/nova master: Test ceph-multistore with a real image https://review.opendev.org/c/openstack/nova/+/860864 | 14:54 |
| Kirill_ | Hi, do we have a meeting today? | 15:01 |
| elodilles | Kirill_: if i'm not mistaken we will have @ 16:00 UTC | 15:03 |
| Kirill_ | got it, thanks | 15:03 |
| opendevreview | sean mooney proposed openstack/nova master: Support multiple config file with mod_wsgi https://review.opendev.org/c/openstack/nova/+/864014 | 15:16 |
| bauzas | reminder : nova meeting in 34 min | 15:26 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/train: Adds a repoducer for post live migration fail https://review.opendev.org/c/openstack/nova/+/863806 | 15:42 |
| han-guangyu | Hello, My name is Han Guangyu. | 15:43 |
| han-guangyu | Currently, I work in a startup operating system company. Some time ago, the work I participated in : building rpm package for openstack, deploying some small private clouds using openstack, dealing with usage problems encountered by customers. | 15:43 |
| han-guangyu | I have some basic open source community participation experience. Such as, I have been involved the work of OpenStack SIG of openEuler community(https://www.openeuler.org/en/ A OS community). The basic community process and the use of tools such as git and gerrit are ok for me | 15:43 |
| han-guangyu | Now, My job allowed me to transition into an openstack developer. So, I hope and have time to get involved in the openstck community. | 15:43 |
| han-guangyu | But because there is no accumulation, I now mainly rely on self-study. At present, I have initially debugged the process of creating an instance, and I have an understanding of the whole, but I have not gone into the details. Earlier I assigned myself a low-hanging-frout bug, and I wanted to finish it (Sorry delay to work it). | 15:43 |
| han-guangyu | I have an understanding of the basic syntax of Python, but I have only written some simple scripts and have not actually participated in the development of large-scale python projects. So, Nova is complicated, for me. | 15:43 |
| han-guangyu | Just an introduction to myself. And I would like to ask, for someone without much python experience, do you have any suggestions for self-learning nova source code? | 15:44 |
| bauzas | #startmeeting nova | 16:00 |
| opendevmeet | Meeting started Tue Nov 8 16:00:04 2022 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| opendevmeet | The meeting name has been set to 'nova' | 16:00 |
| bauzas | hello folks, welcome in the nova meeting | 16:00 |
| gibi | o/ | 16:00 |
| Kirill_ | 0/ | 16:00 |
| elodilles | o/ | 16:00 |
| * gibi is distracted | 16:00 | |
| gmann | o/ | 16:00 |
| bauzas | k, let's start | 16:01 |
| auniyal | O/ | 16:01 |
| bauzas | #topic Bugs (stuck/critical) | 16:01 |
| bauzas | #topic Bugs (stuck/critical) | 16:01 |
| bauzas | #info No Critical bug | 16:01 |
| bauzas | #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 5 new untriaged bugs (-3 since the last meeting) | 16:01 |
| bauzas | #info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster | 16:01 |
| bauzas | thanks gibi and others for having looked at the bugs | 16:01 |
| bauzas | any bug to discuss ? | 16:02 |
| bauzas | looks not | 16:03 |
| Uggla | o/ | 16:03 |
| han-guangyu | o/ | 16:03 |
| bauzas | melwitt: could you be using the bug baton next week ? | 16:03 |
| gibi | bauzas: yeah I did some triage last week | 16:04 |
| gibi | not much this week thouh | 16:04 |
| gibi | though | 16:04 |
| bauzas | cool | 16:04 |
| bauzas | ok, I'll ask again to melwitt to know if she can help for bug triage this week | 16:05 |
| bauzas | moving on | 16:05 |
| bauzas | #topic Gate status | 16:05 |
| bauzas | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | 16:05 |
| bauzas | #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&pipeline=periodic-weekly Nova&Placement periodic jobs status | 16:05 |
| bauzas | #info Please look at the gate failures and file a bug report with the gate-failure tag. | 16:05 |
| bauzas | #info STOP DOING BLIND RECHECKS aka. 'recheck' https://docs.openstack.org/project-team-guide/testing.html#how-to-handle-test-failures | 16:05 |
| bauzas | as you can see, no issues for the CI periodic jobs (except for fips) | 16:06 |
| bauzas | I need to modify the job to have more 30 mins | 16:06 |
| bauzas | *deadline | 16:06 |
| bauzas | any concern ? | 16:07 |
| bauzas | looks not | 16:07 |
| bauzas | moving on again | 16:07 |
| bauzas | #topic Release Planning | 16:07 |
| bauzas | #link https://releases.openstack.org/antelope/schedule.html | 16:08 |
| bauzas | #info Antelope-1 is planned in 1 week | 16:08 |
| bauzas | sl | 16:08 |
| bauzas | so | 16:08 |
| bauzas | when we should have a specs review day ? | 16:08 |
| bauzas | next tuesday works ? | 16:08 |
| sean-k-mooney | proably yes | 16:09 |
| sean-k-mooney | ill be around and hopefully have more time to review next week | 16:09 |
| bauzas | ok, anyone having an issue ? | 16:10 |
| bauzas | for next tuesday | 16:10 |
| gibi | works for me | 16:10 |
| bauzas | ok | 16:11 |
| bauzas | #agreed We will have have a spec review day on Nov 15th, please prepare your specs before this day and make sure you can be around on this day if you have reviews | 16:12 |
| bauzas | #action bauzas to write an openstack-discuss email for this | 16:12 |
| bauzas | #topic Review priorities | 16:12 |
| bauzas | #link https://review.opendev.org/q/status:open+(project:openstack/nova+OR+project:openstack/placement+OR+project:openstack/os-traits+OR+project:openstack/os-resource-classes+OR+project:openstack/os-vif+OR+project:openstack/python-novaclient+OR+project:openstack/osc-placement)+(label:Review-Priority%252B1+OR+label:Review-Priority%252B2) | 16:13 |
| bauzas | as you can see, now we have more review-prio usage from contributors :) | 16:14 |
| bauzas | <3 | 16:14 |
| bauzas | so, please, look at those when you review | 16:14 |
| bauzas | #info As a reminder, cores eager to review changes can +1 to indicate their interest, +2 for committing to the review | 16:14 |
| bauzas | any question here ? | 16:14 |
| sean-k-mooney | its not qiute what i was suggestign for the usage but ok | 16:15 |
| sean-k-mooney | +1 was commit to review +2 was asking for other cores to also review | 16:15 |
| sean-k-mooney | but either woks | 16:15 |
| bauzas | k | 16:16 |
| bauzas | at least contributors can +1 if they want core reviews for some change they reviewed (not for their own changes of course) | 16:17 |
| bauzas | moving on ? | 16:18 |
| bauzas | #topic Stable Branches | 16:18 |
| bauzas | elodilles : your time | 16:18 |
| elodilles | #info wallaby transitioned to Extended Maintenance ( https://review.opendev.org/c/openstack/releases/+/862298 ) | 16:18 |
| elodilles | and we have released a final wallaby release | 16:19 |
| elodilles | #info stable branches state should be OK (except stein and older) | 16:19 |
| elodilles | #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci | 16:19 |
| elodilles | and last but not least: | 16:19 |
| elodilles | as said 2 weeks ago: stein, rocky and queens EOL'ing patch was proposed: https://review.opendev.org/862520 + mail was sent to ML https://lists.openstack.org/pipermail/openstack-discuss/2022-October/030980.html | 16:20 |
| elodilles | as I see no mail replies arrived on the thread | 16:20 |
| elodilles | so I propose to continue with the EOL'ing | 16:20 |
| bauzas | yup, agreed | 16:20 |
| bauzas | I just +1d the change | 16:20 |
| elodilles | bauzas: thanks! | 16:21 |
| bauzas | we agreed on it during the PTG | 16:21 |
| bauzas | and given none has concerns by the email, that means we can EOL | 16:21 |
| bauzas | elodilles: I guess you can now remove the -W | 16:21 |
| elodilles | removed it now | 16:22 |
| bauzas | thbanks | 16:22 |
| bauzas | anything else to discuss about the stable branches ? | 16:23 |
| elodilles | nothing from me | 16:23 |
| bauzas | ok, last topic then | 16:24 |
| bauzas | #topic Open discussion | 16:24 |
| bauzas | we have an item today | 16:25 |
| bauzas | Kirill_ : vnc console for baremetal. new spec: https://review.opendev.org/c/openstack/nova-specs/+/863773 | 16:25 |
| bauzas | I guess you want reviews ? | 16:25 |
| Kirill_ | hi! yes. also i've emplemented it) | 16:26 |
| bauzas | Kirill_: OK, then no worries, we'll look at your spec at least on next Tuesday | 16:27 |
| sean-k-mooney | so you used to be able to do this in the past | 16:27 |
| bauzas | but if you can discuss about the design, sure, let us knwo | 16:27 |
| sean-k-mooney | with the ipmi driver and vncinabox or something like that | 16:27 |
| bauzas | if you *want | 16:27 |
| sean-k-mooney | shellinabox | 16:28 |
| sean-k-mooney | that was the ironic optional component | 16:28 |
| sean-k-mooney | https://docs.openstack.org/ironic/latest/admin/console.html | 16:28 |
| sean-k-mooney | if you use that you can enabel the console to work in horizong today | 16:29 |
| Kirill_ | i work on idrac servers+adding ilo console | 16:29 |
| Kirill_ | but the main gaul is | 16:29 |
| Kirill_ | new graphical console interface | 16:29 |
| Kirill_ | i also emplemented it. but will create mr after approve of this spec | 16:30 |
| Kirill_ | in my spec i added all links. also link to new graphical console interface approved spec | 16:31 |
| sean-k-mooney | ok i would liek to see some context in the spec why the existing horizon integration which used to work with nova provisiond server no longer works | 16:31 |
| sean-k-mooney | ok the reaons im asking this i si used ot have ironic deployed with the geniric ipmi driver and shellinabox at home a few years ago and it worked fine in horizon | 16:32 |
| sean-k-mooney | useing a difffent ironic driver ideally shoudl not requrie changed to nova | 16:32 |
| sean-k-mooney | so if we add this new feature i would hope it will work for all ironic drivers | 16:32 |
| Kirill_ | for vnc | 16:32 |
| Kirill_ | we need to create a handshake | 16:32 |
| Kirill_ | for this handshake we need a password from ironic | 16:33 |
| sean-k-mooney | oh the old solution used the serial console proxy | 16:33 |
| sean-k-mooney | not the novnc proxy | 16:33 |
| sean-k-mooney | that the delta | 16:33 |
| sean-k-mooney | ok makes sense | 16:33 |
| Kirill_ | we need a graphical console not serial | 16:33 |
| sean-k-mooney | ack ya just was trying to figure out why this is fiddernt then https://docs.openstack.org/ironic/latest/admin/console.html | 16:34 |
| sean-k-mooney | its the grapical console not serial | 16:34 |
| Kirill_ | it is not the same console as vnc | 16:35 |
| bauzas | sean-k-mooney: Kirill_: could you maybe discuss this in the spec then ? | 16:36 |
| bauzas | or after the meeting ? | 16:36 |
| Kirill_ | ++ | 16:36 |
| bauzas | I'm about to end the meeting | 16:36 |
| bauzas | so you can continue to discuss in the IRC channel after | 16:36 |
| bauzas | ok, anyone else having an item to discuss ? | 16:37 |
| Kirill_ | just to sum up. on 15th November we will review all specs (and mine too)?))) | 16:37 |
| bauzas | yes | 16:38 |
| Kirill_ | got it, thanks | 16:38 |
| bauzas | ok, then let's close the meeting | 16:39 |
| bauzas | thanks all | 16:39 |
| bauzas | #endmeeting | 16:39 |
| opendevmeet | Meeting ended Tue Nov 8 16:39:45 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:39 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2022/nova.2022-11-08-16.00.html | 16:39 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2022/nova.2022-11-08-16.00.txt | 16:39 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2022/nova.2022-11-08-16.00.log.html | 16:39 |
| elodilles | thanks bauzas o/ | 16:40 |
| opendevreview | Dan Smith proposed openstack/nova master: Test ceph-multistore with a real image https://review.opendev.org/c/openstack/nova/+/860864 | 17:12 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/train: Adds a repoducer for post live migration fail https://review.opendev.org/c/openstack/nova/+/863806 | 19:05 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/train: [compute] always set instance.host in post_livemigration https://review.opendev.org/c/openstack/nova/+/864055 | 19:05 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/train: Adds a repoducer for post live migration fail https://review.opendev.org/c/openstack/nova/+/863806 | 20:08 |
| opendevreview | Amit Uniyal proposed openstack/nova stable/train: [compute] always set instance.host in post_livemigration https://review.opendev.org/c/openstack/nova/+/864055 | 20:08 |
| *** dasm is now known as dasm|off | 22:28 | |
| clarkb | Hello nova. I've been testing server rescue behavior recently and run into two different interesting behaviors. The first is when rescuing a non bfv instance if the rootfs label configured in grub and fstab is the same between the rescue image and the instance being recsued you can boot into the rescue image kernel but have the rescued instance / mounted | 23:00 |
| clarkb | This is problematic because if the problem is say in systemd init steps you'd still be broken in the rescued setup. Additionally it may not always be the case that the kernel and your rescued filesystem are compatbile enough to boot | 23:00 |
| clarkb | That said I'm not sure if nova can do anything to deal with this problem direclty. I suspect one of the best options is for clouds to have a purpose built rescue image that isn't likely to collide in this way | 23:01 |
| clarkb | The other issue is when doing rescue on a bfv instance the api accepts the request (as long as yo uset the api version high enough) but then the instance promptly goes into an error state with Driver Error: Cannot access storage file and no such file or directory with a path. | 23:02 |
| clarkb | It almost looks like nova / libvirt are looking for a disk file rather than looking for the volume via whatever the volume provider mechansim is | 23:02 |
| clarkb | I don't have enough access on the cloud side to debug this further so I'm not sure if this is a nova issue or a cloud configuration issue etc. | 23:03 |
| clarkb | It does make me wonder if we've got any rescue testing to ensure this generally works? Also, should we try to write more docs on how to ensure rescues work? | 23:03 |
| clarkb | I'm coming at this as an end user wishing this functioned bette rand wondering what I/we can do to get there. But I lack a lot of background and knowledge on the inner working here :) | 23:04 |
| clarkb | separately, I do wonder if it makes more sense for bfv rescue to be a process more like "stop instnace, detach volume from instance, attach volume to another instance, make changes, reattach to original instance and start original instance" but this fails because you can't detach a root device | 23:05 |
| opendevreview | melanie witt proposed openstack/nova master: DNM testing images_type = raw with resize enabled https://review.opendev.org/c/openstack/nova/+/862416 | 23:29 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!