Tuesday, 2019-12-10

*** __ministry has joined #senlin04:33
*** sapd1 has joined #senlin06:04
*** sapd1 has quit IRC07:14
rm_workyeah, it wasn't...13:44
rm_workso i am unclear as to WHY exactly it happened13:44
rm_workbut we managed to get it to apply13:44
rm_workanyway, different subject -- updating to use the new services!13:46
rm_workone service went away, and two new ones were created? or JUST two new ones I guess?13:46
*** jmlowe has joined #senlin16:00
rm_workAHHHH and FYI, it was a downgrade -- we tried to run an older version (migration 13) on a DB that was at migration 15. that's where the error message came from16:10
rm_workand someone solved it by just manually updating the number in the table to 13... which i just discovered when i actually tried to upgrade correctly and it exploded because the migrations wouldn't apply T_T16:10
rm_work...16:28
rm_workok so have you seen this one?16:28
rm_workRESP BODY: {"code": 500, "error": {"code": 500, "message": "wrap_socket() got an unexpected keyword argument '_context'", "type": "TypeError"}, "explanation": "The server has either erred or is incapable of performing the requested operation.", "title": "Internal Server Error"}16:28
rm_workthat's from `openstack cluster list --debug`16:28
rm_workis that from me having too old of a client or something?16:35
rm_work2019-12-10 16:37:30,778 DEBUG [senlin.api.common.serializers] /opt/openstack/venv/senlin/lib/python3.7/site-packages/senlin/api/common/serializers.py:to_json:87 JSON response : {"code": 500, "error": {"code": 500, "message": "wrap_socket() got an unexpected keyword argument '_context'", "type": "TypeError"}, "explanation": "The server has either erred or is incapable of performing the requested operation.", "title": "Internal Server16:38
rm_workError"}16:38
rm_workhmmm no16:44
rm_workthis may be an error on the senlin side? just doing a basic curl16:44
rm_workeandersson / dtruong ^^16:45
rm_workcurl "http://mycloud/v1/clusters" -H "X-Auth-Token: <mytoken>"16:46
rm_workthat's all I'm doing to get that 50016:46
*** jmlowe has quit IRC16:50
eanderssonnever seen that one rm_work16:53
rm_workhmm16:53
eanderssonwhat version is this rm_work?16:54
rm_workmaster16:55
rm_workas of an hour ago or so16:55
eanderssonSo we no longer test regular api endpoint, we use the wsgi one for CI16:55
rm_workhmmm16:55
rm_workffff16:55
rm_workk16:55
rm_workcould be related16:55
eanderssonbut I don't see how wrap_socket could be passed _context16:56
eanderssonhttps://github.com/openstack/senlin/blob/71d9a66abe93049a91d433f805c045abe135303a/senlin/api/common/wsgi.py#L28416:56
eanderssonI have a PR to rip some of that out and replace it with oslo.service wsgi impl16:57
rm_workyeah literally no idea16:58
rm_workby keyword no less?16:58
eanderssonHonestly never tried enabling ssl without using wsgi16:59
eanderssonso could be an old bug16:59
rm_workerr actually16:59
rm_worki shouldn't be running in SSL?16:59
eanderssonIt’s the only reason that path would be called right?17:00
rm_workthe api-runner sits behind Apache17:00
rm_workyeah i am thinking it must be in a different piece17:00
rm_workmaybe external17:01
rm_worklike, calling out to keystone to verify a token or something17:01
rm_work?17:01
eanderssonYea maybe17:01
rm_workwish i had more stacktrace17:01
rm_workbut this serializer kinda eats it17:01
rm_workhttps://github.com/eventlet/eventlet/issues/52617:16
rm_workhmm17:16
rm_workok so17:16
rm_workwe need py3.617:16
rm_worknot py3.717:16
rm_worksenlin is not yet compatible with py37?17:16
rm_workthough it was fixed in late 2018 so not sure why it'd still be an issue now17:17
rm_workyou guys use eventlet I guess? :/17:18
rm_workeventlet is poison :(17:18
rm_workhttps://review.opendev.org/#/c/454873/17:19
rm_workwe even have https://review.opendev.org/#/c/462334/2/octavia/hacking/checks.py@25717:19
rm_workeandersson: ^^17:19
eanderssonI thought we had a python 3.7 test17:23
eanderssonYou using the wsgi endpoint right? Or senlin-api?17:23
rm_worksenlin-api i think17:24
rm_workbut eventlet is pulled in multiple places it looks like... and we may run into issues from that too17:24
dtruongThe fix for that issue is in eventlet >= 0.25.017:25
dtruongSo double check you are using that version or higher17:25
rm_workeventlet==0.25.117:28
rm_workbut17:28
rm_worki followed the advice in the last post of that thread and installed pyopenssl==19.1.017:28
rm_workand it seems to work :D17:28
eanderssonInteresting17:41
rm_workit appears that is the highest version of pyopenssl anyway17:42
rm_workso i just unpinned17:42
rm_workwhy ya'll gotta use eventlet at all? :D17:49
eanderssonSomeone made that decision a long time ago :p[18:19
rm_worklet's see ... how bored am I18:23
rm_workone of my favorite pastimes *is* finding things I hate and burning them with righteous fire18:23
rm_workeandersson: where's the patch you were talking about?18:43
eanderssonSorry I haven't uploaded it yet. I am just removing the custom wsgi implementation and replacing it with oslo.service wsgi18:44
eanderssonSimilar to what I did for Designate18:44
rm_workah yeah i saw your designate patch, that was just a devstack thing?18:48
rm_workor was there another one18:48
rm_workhttps://review.opendev.org/#/c/454873/9/octavia/cmd/octavia_worker.py18:49
rm_workplease avoid oslo.service <_<18:49
rm_workwe killed off oslo.service almost 3 years ago :D18:50
rm_worklook at cotyledon18:50
rm_workand then remove any references to eventlet :D18:51
rm_workeandersson: ^^ pretty please18:51
rm_workah hmm, i guess for the wsgi endpoint, MAYBE that's ok? as long as it doesn't import eventlet anywhere18:53
rm_workwe used `from wsgiref import simple_server`18:53
rm_workfor our wsgi definition18:53
rm_workhttps://github.com/openstack/octavia/blob/master/octavia/cmd/api.py18:54

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!