| openstackgerrit | Merged openstack/networking-powervm master: Dropping the py35 testing https://review.opendev.org/654647 | 00:45 |
|---|---|---|
| *** ricolin has joined #openstack-powervm | 02:06 | |
| *** dikonoor has joined #openstack-powervm | 02:51 | |
| *** vijayendra has joined #openstack-powervm | 05:22 | |
| *** aarora06 has joined #openstack-powervm | 05:42 | |
| openstackgerrit | Merged openstack/ceilometer-powervm master: Dropping the py35 testing https://review.opendev.org/654646 | 07:20 |
| openstackgerrit | Le Hou proposed openstack/networking-powervm master: Use opendev repository https://review.opendev.org/655342 | 07:35 |
| *** dikonoor has quit IRC | 08:01 | |
| openstackgerrit | Le Hou proposed openstack/ceilometer-powervm master: Use opendev repository https://review.opendev.org/654838 | 10:14 |
| *** aarora06_ has joined #openstack-powervm | 10:21 | |
| openstackgerrit | Le Hou proposed openstack/networking-powervm master: Use opendev repository https://review.opendev.org/655342 | 10:26 |
| openstackgerrit | Le Hou proposed openstack/ceilometer-powervm master: Use opendev repository https://review.opendev.org/654838 | 10:28 |
| *** aarora06_ has quit IRC | 10:31 | |
| *** aarora06 has quit IRC | 11:39 | |
| *** aarora06 has joined #openstack-powervm | 11:48 | |
| *** aarora06 has quit IRC | 11:52 | |
| openstackgerrit | Merged openstack/ceilometer-powervm master: Use opendev repository https://review.opendev.org/654838 | 12:19 |
| *** dikonoor has joined #openstack-powervm | 12:36 | |
| *** aarora06 has joined #openstack-powervm | 12:46 | |
| *** aarora06 has quit IRC | 12:53 | |
| *** irclogbot_2 has quit IRC | 12:55 | |
| *** irclogbot_2 has joined #openstack-powervm | 12:56 | |
| *** altlogbot_2 has quit IRC | 12:57 | |
| *** altlogbot_2 has joined #openstack-powervm | 12:58 | |
| *** vijayendra has quit IRC | 14:02 | |
| *** ricolin has quit IRC | 15:17 | |
| *** mujahidali has joined #openstack-powervm | 15:25 | |
| *** ricolin has joined #openstack-powervm | 15:28 | |
| mujahidali | efried: Hi | 15:30 |
| efried | Hi mujahidali | 15:30 |
| mujahidali | Powervm-CI jobs are failing for most of the time and when I checked the logs every failure is for some different test case but the most common error message I saw is “exceeded maximum number of retries, exhausted all host”. | 15:30 |
| mujahidali | Do you know is there any recent patchset in nova that may be causing the error? | 15:30 |
| efried | mujahidali: That's in the n-sch log? | 15:31 |
| efried | I don't know of anything significant. When was the last time it succeeded? | 15:31 |
| mujahidali | the failures are intermittent. | 15:31 |
| efried | my favorite :) | 15:32 |
| mujahidali | dikonoor looked into the nova logs and she was supecting that it might be some connection issue with io, so we are trying to upgrade the VIOS and see how that goes. | 15:33 |
| efried | okay. | 15:33 |
| efried | Otherwise, the first thing I would look for is placement logs for GET /allocation_candidates requests. | 15:34 |
| efried | If that returns a small payload, like 50-odd bytes, it means no candidates were returned. | 15:34 |
| efried | which means placement didn't think you had any hosts with enough resources. | 15:35 |
| efried | But the message you posted, about retries, implies that you got further than that, and selected a host, but the spawn failed on that host for whatever reason. The scheduler retries with the next available host (will there be more than one?) for a maximum of three. | 15:36 |
| efried | so you'll want to check the nova-compute logs to see if the spawn request got that far, and if it's failing there, why. | 15:36 |
| mujahidali | I will check that. | 15:38 |
| mujahidali | do you want to share any update about nova that we should take care ? | 15:39 |
| mujahidali | efried: ^ | 15:43 |
| *** amansi26 has joined #openstack-powervm | 15:44 | |
| efried | mujahidali: If nova did something that broke you, then yes, I definitely want to know. | 15:48 |
| efried | We try to be courteous and let oot drivers know when we do something like that. | 15:48 |
| efried | there are a couple of changes in flight that modify ComputeDriver method signatures, but I don't think any of them have landed lately. | 15:49 |
| mujahidali | PowerVM-CI was down, so we might have missed to test them. Do you have gerrit request for that change ? | 15:50 |
| efried | https://review.opendev.org/642590 | 15:54 |
| efried | https://review.opendev.org/636699 | 15:54 |
| efried | https://review.opendev.org/613039 | 15:54 |
| efried | mujahidali ^ that's everything since at least 4Q18 | 15:54 |
| mujahidali | efried: thank you :) | 15:54 |
| efried | FYI, got there by asking gerrit for changes that touched powervm/driver.py | 15:54 |
| efried | https://review.opendev.org/#/q/project:openstack/nova+path:%255Enova/virt/powervm/driver.py | 15:54 |
| mujahidali | I will retry and see how it goes. | 15:55 |
| efried | good luck | 15:55 |
| mujahidali | thanks | 15:55 |
| *** efried is now known as efried_rollin | 16:04 | |
| *** aarora06 has joined #openstack-powervm | 16:08 | |
| *** mujahidali has quit IRC | 16:08 | |
| *** altlogbot_2 has quit IRC | 16:09 | |
| *** altlogbot_1 has joined #openstack-powervm | 16:12 | |
| *** aarora06 has quit IRC | 16:14 | |
| *** ricolin has quit IRC | 16:20 | |
| *** altlogbot_1 has quit IRC | 16:43 | |
| *** altlogbot_2 has joined #openstack-powervm | 16:44 | |
| *** amansi26 has quit IRC | 16:45 | |
| *** altlogbot_2 has quit IRC | 16:53 | |
| *** dikonoor has quit IRC | 16:54 | |
| *** altlogbot_3 has joined #openstack-powervm | 16:54 | |
| *** openstackgerrit has quit IRC | 18:57 | |
| *** efried_rollin is now known as efried | 20:14 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!