Friday, 2022-04-29

dtruong__ministry From looking at the logs, somehow the health manager got into a bad state.  The health manager calls load_runtime_registry every 60secs here: https://github.com/openstack/senlin/blob/3b0f21972be0bb067e8c4391a6b77aa8815a0ca2/senlin/health_manager/service.py#L10305:13
dtruongload_runtime_registry checks if the health checks have have been claimed by an engine by checking the DB: https://github.com/openstack/senlin/blob/master/senlin/engine/health_manager.py#L75305:14
dtruongAnother engine should have claimed those health checks so db_registries should be empty and it should not print out that loading cluster on https://github.com/openstack/senlin/blob/master/senlin/engine/health_manager.py#L77305:16
eanderssonDo you have any idea on how we could reproduce this_05:17
dtruongIf this case happens, please check HealthRegistry table in the DB and see if there are multiple entries for the same health checks05:17
dtruongeandersson: No, I don't know how this could be happen.  Must be some sort of corner case bug.05:18
eanderssonYea - that makes sense. I tried to break it in every manner possible and wasnt able to make that happen.05:29
eandersson__ministry: Could you open a bug and compile as much information as possible_06:42
opendevreviewVishal Manchanda proposed openstack/senlin-dashboard master: Address RemovedInDjango40Warning  https://review.opendev.org/c/openstack/senlin-dashboard/+/83993012:01
opendevreviewVishal Manchanda proposed openstack/senlin-dashboard master: Address RemovedInDjango40Warning  https://review.opendev.org/c/openstack/senlin-dashboard/+/83993016:05

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!