Thursday, 2017-04-27

*** shewless has quit IRC00:29
*** hoangcx has joined #openstack-dns00:45
*** cuongnv has joined #openstack-dns00:56
*** trugnv has joined #openstack-dns01:07
*** EricGonczer_ has quit IRC01:46
*** egonzalez has quit IRC01:54
*** jhfeng has joined #openstack-dns01:55
*** zhurong has joined #openstack-dns02:02
*** zhurong has quit IRC02:18
*** zhurong has joined #openstack-dns02:44
*** chlong has joined #openstack-dns02:48
*** zhurong has quit IRC03:42
*** jhfeng has quit IRC04:23
*** castlemilk has joined #openstack-dns05:23
castlemilkAre there any common tools out there for pruning designate's DB? given that when managed=true you can't do anything.05:24
castlemilkI'm having issues where some records arent being correctly deleted when the instance is rapidly created then deleted05:24
castlemilkim currently rolling my own solution which is pruning the mysql DB, not sure if this is best practice05:25
*** richm has quit IRC05:44
openstackgerritHieu LE proposed openstack/designate master: Refactor NSD4 UT for using upstream eventlet  https://review.openstack.org/46040406:00
*** castlemi_ has joined #openstack-dns06:04
*** castlemilk has quit IRC06:07
*** fandi has joined #openstack-dns06:12
*** zhurong has joined #openstack-dns06:12
*** castlemilk has joined #openstack-dns06:20
*** castlemi_ has quit IRC06:20
*** mkoderer_ has joined #openstack-dns06:25
*** tpatzig_ has joined #openstack-dns06:25
*** mkoderer_ has quit IRC06:27
*** tpatzig_ has quit IRC06:27
*** pcaruana has joined #openstack-dns06:44
*** hoangcx_ has joined #openstack-dns06:52
openstackgerritHieu LE proposed openstack/designate master: Refactor NSD4 UT for using upstream eventlet  https://review.openstack.org/46040406:53
*** hoangcx has quit IRC06:54
*** zhurong has quit IRC07:02
*** zhurong has joined #openstack-dns07:06
*** abalutoiu__ has quit IRC07:21
*** hoangcx has joined #openstack-dns07:27
*** hoangcx_ has quit IRC07:28
*** zhurong has quit IRC07:37
*** amoralej|off is now known as amoralej07:39
*** abalutoiu has joined #openstack-dns07:42
*** abalutoiu has quit IRC07:51
*** hoangcx_ has joined #openstack-dns07:55
*** hoangcx has quit IRC07:57
*** abalutoiu has joined #openstack-dns08:07
*** hoangcx has joined #openstack-dns08:18
*** zhurong has joined #openstack-dns08:19
*** hoangcx_ has quit IRC08:20
castlemilkhow to resolve record conflics/inconsistencies? any tooling available?08:34
*** MarkBaker has joined #openstack-dns08:37
*** hieulq has quit IRC08:40
*** hoangcx has quit IRC08:42
*** cuongnv has quit IRC08:42
*** trugnv has quit IRC08:42
*** EricGonczer_ has joined #openstack-dns08:42
*** EricGonczer_ has quit IRC08:47
*** EricGonczer_ has joined #openstack-dns08:53
*** MarkBaker has quit IRC09:16
*** sgr7 has joined #openstack-dns09:25
*** stk_ has joined #openstack-dns09:26
*** egonzalez has joined #openstack-dns09:27
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/designate master: [docs] Rearranges index.rst to build nicer with the theme  https://review.openstack.org/44607909:37
asettleodyssey4me: ya bored? :p09:39
*** sgr7 has quit IRC09:40
odyssey4menope - just doing what I can to move things along09:40
odyssey4methe docs build for the patch no longer had data, so I rebased to ensure it rebuilds09:40
mugsiecastlemilk: you can use the openstack cli with --all-projects and --edit-managed-records to change those records09:54
castlemilki get this strange error when using --edit-managed09:55
castlemilkHeader value True must be of type str or bytes, not <type 'bool'>09:55
castlemilkrunning on RDO09:56
castlemilkdesignate --version09:56
castlemilkdesignate 2.3.009:56
mugsieah, they have an old version09:56
castlemilkis 2.3.0 the old version?09:57
mugsieyeah09:57
mugsie2.6.0 is the new one09:57
*** kiall has quit IRC10:00
*** stk_ has quit IRC10:01
*** kiall has joined #openstack-dns10:01
*** kiall has joined #openstack-dns10:01
*** egonzalez has quit IRC10:03
*** egonzalez has joined #openstack-dns10:04
castlemilk+---------+------------------------------------+10:06
castlemilk| Field   | Value                              |10:06
castlemilk+---------+------------------------------------+10:06
castlemilk| Code    | 400                                |10:06
castlemilk| Type    | bad_request                        |10:07
castlemilk| Message | Managed records may not be deleted |10:07
castlemilk+---------+------------------------------------+10:07
castlemilkwith 2.6.0 and --edit-manged flag, have you seen that before?10:07
castlemilk@mugsie10:07
mugsiewait, wut10:12
mugsieeh, let me look10:12
castlemilkin debug i see:10:15
*** richm has joined #openstack-dns10:15
mugsieand you set --edit-managed ?10:15
*** hieulq has joined #openstack-dns10:15
castlemilk-H "X-Designate-Edit-Managed-Records: true"10:16
castlemilkin the debug logs10:16
mugsiecan you get logs from the API server ?10:16
castlemilkyea10:16
castlemilk127.0.0.1 - - [27/Apr/2017 20:15:43] "DELETE /v1/domains/055b9a01-0d8f-49af-b53c-3eb733e1601a/records/03947911-4005-4f95-8bc1-a28aa9d41ce0 HTTP/1.1" 400 351 0.03759510:17
castlemilkis it because its using v1?10:17
mugsieit shouldnt be10:17
mugsiecan you see if you can find the context for that request ?10:18
*** hieulq has quit IRC10:18
castlemilkumm how would that be done?10:18
mugsie#it should be logged just before that line10:18
castlemilkoh ok10:19
mugsieit will be a json object, with things like "project_id", "roles" etc10:19
mugsieand a property "edit_managed_records"10:19
castlemilk2017-04-27 20:15:43.359 11741 DEBUG eventlet.wsgi [-] (11741) accepted ('10.0.0.10', 59472) server /usr/lib/python2.7/site-packages/eventlet/wsgi.py:86710:20
castlemilk2017-04-27 20:15:43.361 11741 INFO eventlet.wsgi [-] 10.0.0.10 - - [27/Apr/2017 20:15:43] "GET / HTTP/1.1" 200 574 0.00105710:20
castlemilk2017-04-27 20:15:43.363 11741 DEBUG eventlet.wsgi [-] (11741) accepted ('127.0.0.1', 51800) server /usr/lib/python2.7/site-packages/eventlet/wsgi.py:86710:20
castlemilk2017-04-27 20:15:43.366 11741 DEBUG keystoneauth.session [-] REQ: curl -g -i -X GET http://10.0.0.9:35357/v3/auth/tokens -H "X-Subject-Token: {SHA1}9f43ea1d1523b7b1f91b7d9a597b54c9dbc9e3ce" -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}e09d3b92ff2f111a709b352bc98503865bb48d32" _http_log_request /usr/lib/python2.7/site-packages/keystoneauth1/session.py:34710:20
castlemilk2017-04-27 20:15:43.368 11741 DEBUG requests.packages.urllib3.connectionpool [-] Resetting dropped connection: 10.0.0.9 _get_conn /usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:24710:20
castlemilk2017-04-27 20:15:43.734 11741 DEBUG requests.packages.urllib3.connectionpool [-] http://10.0.0.9:35357 "GET /v3/auth/tokens HTTP/1.1" 200 8767 _make_request /usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:40010:20
castlemilk2017-04-27 20:15:43.736 11741 DEBUG keystoneauth.session [-] RESP: [200] Date: Thu, 27 Apr 2017 10:15:43 GMT Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips mod_wsgi/3.4 Python/2.7.5 X-Subject-Token: {SHA1}9f43ea1d1523b7b1f91b7d9a597b54c9dbc9e3ce Vary: X-Auth-Token x-openstack-request-id: req-73310845-0ecb-4af7-b8a0-2d37ad021b02 Content-Length: 8767 Keep-Alive: timeout=5, max=100 Connection: Keep-Ali10:20
castlemilkve Content-Type: application/json10:20
castlemilkRESP BODY: {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "0c591e64e85846ae9b2f2ae88be5982a", "name": "admin"}, {"id": "ae9be7e8b27641f4a1f6c87d43c43eb5", "name": "heat_stack_owner"}], "expires_at": "2017-04-27T11:15:43.000000Z", "project": {"domain": {"id": "default", "name": "Default"}, "id": "f8e9ed2aaf4c4d44a6b74cdde3faf75d", "name": "admin"}, "catalog": "<removed>", "user10:20
castlemilk": {"domain": {"id": "default", "name": "Default"}, "id": "914e9e9b0e28430bbf5c0641374a9a9c", "name": "admin"}, "audit_ids": ["AEiERr55Tnuj81CJkKHfQw"], "issued_at": "2017-04-27T10:15:43.000000Z"}}10:20
castlemilk _http_log_response /usr/lib/python2.7/site-packages/keystoneauth1/session.py:39510:20
castlemilk2017-04-27 20:15:43.736 11741 DEBUG keystoneauth.session [-] GET call to identity for http://10.0.0.9:35357/v3/auth/tokens used request id req-73310845-0ecb-4af7-b8a0-2d37ad021b02 request /usr/lib/python2.7/site-packages/keystoneauth1/session.py:64010:20
castlemilk2017-04-27 20:15:43.739 11741 DEBUG designate.central.rpcapi [req-61419ef4-7b2a-4819-b398-2c00154c34f3 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] Calling designate.central.find_zones() over RPC wrapped /usr/lib/python2.7/site-packages/designate/loggingutils.py:2410:20
castlemilk2017-04-27 20:15:43.769 11741 INFO designate.api.v1.domains [req-61419ef4-7b2a-4819-b398-2c00154c34f3 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] Retrieved <Zone count:'2' object:'ZoneList'>10:20
castlemilk2017-04-27 20:15:43.770 11741 DEBUG designate.schema.resolvers [req-61419ef4-7b2a-4819-b398-2c00154c34f3 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] Loading remote schema: domain resolve_remote /usr/lib/python2.7/site-packages/designate/schema/resolvers.py:3810:20
*** zhurong has quit IRC10:20
castlemilk2017-04-27 20:15:43.774 11741 INFO eventlet.wsgi [req-61419ef4-7b2a-4819-b398-2c00154c34f3 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] 127.0.0.1 - - [27/Apr/2017 20:15:43] "GET /v1/domains HTTP/1.1" 200 882 0.40973310:20
castlemilk2017-04-27 20:15:43.784 11741 DEBUG designate.central.rpcapi [req-a6b41d7e-6730-4660-b9c4-ebc6c107d22f 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] Calling designate.central.find_zone() over RPC wrapped /usr/lib/python2.7/site-packages/designate/loggingutils.py:2410:20
castlemilk2017-04-27 20:15:43.804 11741 DEBUG designate.central.rpcapi [req-a6b41d7e-6730-4660-b9c4-ebc6c107d22f 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] Calling designate.central.find_record() over RPC wrapped /usr/lib/python2.7/site-packages/designate/loggingutils.py:2410:20
castlemilk2017-04-27 20:15:43.818 11741 INFO eventlet.wsgi [req-a6b41d7e-6730-4660-b9c4-ebc6c107d22f 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] 127.0.0.1 - - [27/Apr/2017 20:15:43] "DELETE /v1/domains/055b9a01-0d8f-49af-b53c-3eb733e1601a/records/03947911-4005-4f95-8bc1-a28aa9d41ce0 HTTP/1.1" 400 351 0.03759510:20
castlemilksorry for the spam10:20
castlemilkbut that is the group within a small timeframe10:20
mugsieweird10:21
castlemilkim not running designate-worker or designate-producer10:23
castlemilkif that impacts anything10:23
openstackgerritHieu LE proposed openstack/designate master: Refactor NSD4 UT for using upstream eventlet  https://review.openstack.org/46040410:23
*** fandi has quit IRC10:24
mugsieno, managed records predate all of that10:25
castlemilkanything else to look at?10:26
castlemilki tried using the openstack client but it has that requests issue with <bool> type10:27
castlemilkwhen using --edit-managed10:27
*** zhurong has joined #openstack-dns10:27
mugsieI am wondering if that is getting set properly in the context10:28
mugsiebecause we should be allowing it10:28
mugsiethe account you are using has admin credentials, right?10:28
castlemilkyea10:28
castlemilkin terms of keystone admin yea10:29
castlemilkis there any additional admin rights i might be missing?10:30
mugsieit depends on if you modified your policy/json10:31
castlemilki am able to delete entire domains etc10:31
mugsieOK, from the snippet ^ it looks like you have the right role10:31
castlemilk   "edit_managed_records" : "rule:admin",10:32
mugsiecan you try an run the curl from the debug, but change 'true' to 'True' for the header ?10:32
*** hieulq has joined #openstack-dns10:33
castlemilkyea i can10:33
castlemilkwhats the best way to dynamically inject a valid token though10:34
*** fandi has joined #openstack-dns10:34
castlemilkif i use the token used in the debug message i get "The request you have made requires authentication."10:34
mugsieif you do openstack token issue10:35
mugsieit will issue one10:35
castlemilki tried this:10:35
castlemilk-H "X-Auth-Token: $(openstack token issue)"10:35
castlemilkcurl -g -i --cacert "" -X DELETE http://127.0.0.1:9001/v1/domains/055b9a01-0d8f-49af-b53c-3eb733e1601a/records/03947911-4005-4f95-8bc1-a28aa9d41ce0 -H "User-Agent: python-designateclient-2.6.0" -H "X-Designate-Edit-Managed-Records: true" -H "Content-Type: application/json" -H "X-Auth-Token: givi$(openstack token issue)"10:36
castlemilkresponse:10:36
castlemilk{"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}10:36
mugsieoh, just run the command, and copy and paste it out10:36
castlemilkcurl -g -i --cacert "" -X DELETE http://127.0.0.1:9001/v1/domains/055b9a01-0d8f-49af-b53c-3eb733e1601a/records/03947911-005-4f95-8bc1-a28aa9d41ce0 -H "User-Agent: python-designateclient-2.6.0" -H "X-Designate-Edit-Managed-Records: True" -H "Content-Type: application/json" -H "X-Auth-Token: gAAAAABZAcnzrXcBCdBaYwFfIPrVZTJWpugkPmzdYI93Ey5yPCySDLmAb_LaMuLRqkuK13Jx3sMh9-ZUOq16okqeQRO1samTN6iLz1Tzlsa4InBzpD10:38
castlemilkC25opF4MtYIELI1e_dJS4Xn0_TGuaTF78CKJ1wtPdDtjZSHIBIvSqrGS-tYGrf0N9CkP4"10:38
castlemilkHTTP/1.1 400 BAD REQUEST10:38
castlemilkContent-Type: application/json10:38
castlemilkContent-Length: 14310:38
castlemilkX-Openstack-Request-Id: req-969c9fba-2c63-4e7b-b8d9-03d29cebf77410:38
castlemilkDate: Thu, 27 Apr 2017 10:37:51 GMT10:38
castlemilk{"message": "Managed records may not be deleted", "code": 400, "type": "bad_request", "request_id": "req-969c9fba-2c63-4e7b-b8d9-03d29cebf774"}10:38
castlemilkthats with True capitalised10:38
mugsiecrap10:39
mugsiewhat version of designate again ?10:39
castlemilkso client is 2.6.010:39
castlemilkbest way to check version of the other components?10:40
mugsieon one of the servers running it designate-api --version will give it10:41
castlemilk[root@controller1 designate]# designate-api --version10:41
castlemilkusage: designate [-h] [--config-dir DIR] [--config-file PATH] [--debug]10:41
castlemilk                 [--log-config-append PATH] [--log-date-format DATE_FORMAT]10:41
castlemilk                 [--log-dir LOG_DIR] [--log-file PATH] [--nodebug]10:41
castlemilk                 [--nouse-syslog] [--nowatch-log-file]10:41
castlemilk                 [--syslog-log-facility SYSLOG_LOG_FACILITY] [--use-syslog]10:41
*** EricGonczer_ has quit IRC10:41
castlemilk                 [--watch-log-file]10:41
*** EricGonc_ has joined #openstack-dns10:41
castlemilkdesignate: error: unrecognized arguments: --version10:41
mugsiecan you get it from yum ?10:42
castlemilkyep10:42
castlemilkhowever i've kinda build this from the github repo as i've made custom notification-handlers10:43
mugsieah10:43
castlemilkbuilt*10:43
mugsiewhat branch on the repo ?10:44
castlemilkwhich i believe was from the newton branch10:44
mugsieOK10:45
mugsieOK, in the curl, remove the X-Designate-Edit-Managed-Records header10:46
mugsieand add ?edit_managed_records=True to the url10:46
*** MarkBaker has joined #openstack-dns10:47
castlemilk[root@controller1 designate]# curl -g -i --cacert "" -X DELETE http://127.0.0.1:9001/v1/domains/055b9a01-0d8f-49af-b53c-3eb733e1601a/records/03947911-4005-4f95-8bc1-a28aa9d41ce0?edit_managed_records=True -H "User-Agent: python-designateclient-2.6.0" "Content-Type: application/json" -H "X-Auth-Token: $(openstack token issue | grep ' id' | awk '{print $4}')"10:51
castlemilkHTTP/1.1 400 BAD REQUEST10:51
castlemilkContent-Type: application/json10:51
castlemilkContent-Length: 14310:51
castlemilkX-Openstack-Request-Id: req-c0fc8d8c-caa0-4c97-968b-ba2a064b4e4010:51
castlemilkDate: Thu, 27 Apr 2017 10:50:52 GMT10:51
castlemilk{"message": "Managed records may not be deleted", "code": 400, "type": "bad_request", "request_id": "req-c0fc8d8c-caa0-4c97-968b-ba2a064b4e40"}curl: (6) Could not resolve host: Content-Type; Name or service not known10:51
*** kiall has quit IRC10:51
*** kiall has joined #openstack-dns10:53
mugsiecan you go to https://github.com/openstack/designate/blob/ca615479a3ab624e391baf483004e72f102e0202/designate/api/middleware.py that file, and add some lines to log data ?10:53
castlemilkyep10:54
mugsienamely, here - https://github.com/openstack/designate/blob/ca615479a3ab624e391baf483004e72f102e0202/designate/api/middleware.py#L127 add a line that logs the "ctxt" variable ?10:54
mugsieand a line to log the request10:54
mugsievariable10:54
mugsieI will brb - need coffee10:55
castlemilkwill it work to add this into the lib in site-packages or do i need to re-build10:58
castlemilkoh heres the version btw:11:02
castlemilkdesignate-4.0.1.dev37-py2.7.egg-info11:02
castlemilkok ive added it just reloading services11:03
castlemilk2017-04-27 21:05:35.070 28562 DEBUG keystoneauth.session [-] REQ: curl -g -i -X GET http://10.0.0.9:35357/v3 -H "Accept: application/json" -H "User-Agent: des11:07
castlemilkignate/4.0.1.dev37 keystonemiddleware.auth_token/4.9.0 keystoneauth1/2.19.0 python-requests/2.12.5 CPython/2.7.5" _http_log_request /usr/lib/python2.7/site-pa11:07
castlemilkckages/keystoneauth1/session.py:34711:07
castlemilk2017-04-27 21:05:35.071 28562 DEBUG requests.packages.urllib3.connectionpool [-] Starting new HTTP connection (1): 10.0.0.9 _new_conn /usr/lib/python2.7/site-11:07
castlemilkpackages/requests/packages/urllib3/connectionpool.py:21211:07
castlemilk2017-04-27 21:05:35.075 28562 DEBUG requests.packages.urllib3.connectionpool [-] http://10.0.0.9:35357 "GET /v3 HTTP/1.1" 200 248 _make_request /usr/lib/pytho11:07
castlemilkn2.7/site-packages/requests/packages/urllib3/connectionpool.py:40011:07
castlemilk2017-04-27 21:05:35.076 28562 DEBUG keystoneauth.session [-] RESP: [200] Date: Thu, 27 Apr 2017 11:05:35 GMT Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips11:07
castlemilk mod_wsgi/3.4 Python/2.7.5 Vary: X-Auth-Token x-openstack-request-id: req-889fa97b-d980-43da-95e1-57063d808f79 Content-Length: 248 Keep-Alive: timeout=5, max=11:07
castlemilk100 Connection: Keep-Alive Content-Type: application/json11:07
castlemilkRESP BODY: {"version": {"status": "stable", "updated": "2016-10-06T00:00:00Z", "media-types": [{"base": "application/json", "type": "application/vnd.openstack11:07
castlemilk.identity-v3+json"}], "id": "v3.7", "links": [{"href": "http://10.0.0.9:35357/v3/", "rel": "self"}]}}11:07
castlemilk _http_log_response /usr/lib/python2.7/site-packages/keystoneauth1/session.py:39511:07
castlemilk2017-04-27 21:05:35.076 28562 DEBUG keystoneauth.session [-] GET call to None for http://10.0.0.9:35357/v3 used request id req-889fa97b-d980-43da-95e1-57063d811:07
castlemilk08f79 request /usr/lib/python2.7/site-packages/keystoneauth1/session.py:64011:07
castlemilk2017-04-27 21:05:35.076 28562 DEBUG keystoneauth.session [-] REQ: curl -g -i -X GET http://10.0.0.9:35357/v3/auth/tokens -H "X-Subject-Token: {SHA1}a6280502a111:07
castlemilkbd3dd447ff760a112382497dfdd322" -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}feb5eb2e6eae17cb544771c46ebc619d311:07
castlemilk2f35d81" _http_log_request /usr/lib/python2.7/site-packages/keystoneauth1/session.py:34711:07
castlemilk2017-04-27 21:05:35.397 28562 DEBUG requests.packages.urllib3.connectionpool [-] http://10.0.0.9:35357 "GET /v3/auth/tokens HTTP/1.1" 200 8767 _make_request /11:07
castlemilkusr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:40011:07
castlemilk2017-04-27 21:05:35.399 28562 DEBUG keystoneauth.session [-] RESP: [200] Date: Thu, 27 Apr 2017 11:05:35 GMT Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips11:07
castlemilk mod_wsgi/3.4 Python/2.7.5 X-Subject-Token: {SHA1}a6280502a1bd3dd447ff760a112382497dfdd322 Vary: X-Auth-Token x-openstack-request-id: req-f5db412b-ec57-4eb8-a11:07
castlemilk1ff-7b3ba2f88fc4 Content-Length: 8767 Keep-Alive: timeout=5, max=99 Connection: Keep-Alive Content-Type: application/json11:07
castlemilkRESP BODY: {"token": {"is_domain": false, "methods": ["password"], "roles": [{"id": "0c591e64e85846ae9b2f2ae88be5982a", "name": "admin"}, {"id": "ae9be7e8b27611:08
castlemilk41f4a1f6c87d43c43eb5", "name": "heat_stack_owner"}], "expires_at": "2017-04-27T12:05:34.000000Z", "project": {"domain": {"id": "default", "name": "Default"},11:08
castlemilk"id": "f8e9ed2aaf4c4d44a6b74cdde3faf75d", "name": "admin"}, "catalog": "<removed>", "user": {"domain": {"id": "default", "name": "Default"}, "id": "914e9e9b0e11:08
castlemilk28430bbf5c0641374a9a9c", "name": "admin"}, "audit_ids": ["KhMC6kVsROuBZ2TREj__aw"], "issued_at": "2017-04-27T11:05:34.000000Z"}}11:08
castlemilk _http_log_response /usr/lib/python2.7/site-packages/keystoneauth1/session.py:39511:08
castlemilk2017-04-27 21:05:35.399 28562 DEBUG keystoneauth.session [-] GET call to identity for http://10.0.0.9:35357/v3/auth/tokens used request id req-f5db412b-ec57-4eb8-a1ff-7b3ba2f88fc4 request /usr/lib/python2.7/site-packages/keystoneauth1/session.py:64011:08
castlemilk2017-04-27 21:05:35.402 28562 DEBUG designate.api.middleware [req-7225b1fe-b78a-4e02-a3e9-7a258c22da03 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] -------------CUSTOM_LOGGING---------------- make_context /usr/lib/python2.7/site-packages/designate/api/middleware.py:12811:08
castlemilk2017-04-27 21:05:35.403 28562 DEBUG designate.api.middleware [req-7225b1fe-b78a-4e02-a3e9-7a258c22da03 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] <designate.context.DesignateContext object at 0x616e2d0> make_context /usr/lib/python2.7/site-packages/designate/api/middleware.py:12911:08
castlemilk2017-04-27 21:05:35.405 28562 DEBUG designate.central.rpcapi [req-7225b1fe-b78a-4e02-a3e9-7a258c22da03 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] Calling designate.central.find_zones() over RPC wrapped /usr/lib/python2.7/site-packages/designate/loggingutils.py:2411:08
castlemilk2017-04-27 21:05:35.406 28562 DEBUG oslo.messaging._drivers.pool [req-7225b1fe-b78a-4e02-a3e9-7a258c22da03 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] Pool creating new connection create /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/pool.py:14311:08
castlemilk2017-04-27 21:05:35.409 28562 DEBUG oslo.messaging._drivers.impl_rabbit [req-7225b1fe-b78a-4e02-a3e9-7a258c22da03 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] [aa4c9343-f670-45a7-9d01-d16161a1c832] Connecting to AMQP server on controller1:5672 __init__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py:56611:08
castlemilk2017-04-27 21:05:35.423 28562 DEBUG oslo.messaging._drivers.impl_rabbit [req-7225b1fe-b78a-4e02-a3e9-7a258c22da03 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] [aa4c9343-f670-45a7-9d01-d16161a1c832] Connected to AMQP server on controller1:5672 via [amqp] client with port 48336. __init__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py:59411:08
castlemilk2017-04-27 21:05:35.424 28562 DEBUG oslo.messaging._drivers.impl_rabbit [req-7225b1fe-b78a-4e02-a3e9-7a258c22da03 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] [aa4c9343-f670-45a7-9d01-d16161a1c832] Queue.declare: reply_20924e84a62e4c11b5c882bc699beebe declare /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/impl_rabbit.py:29911:08
castlemilk2017-04-27 21:05:35.491 28562 INFO designate.api.v1.domains [req-7225b1fe-b78a-4e02-a3e9-7a258c22da03 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] Retrieved <Zone count:'2' object:'ZoneList'>11:08
castlemilk2017-04-27 21:05:35.492 28562 DEBUG designate.schema.resolvers [req-7225b1fe-b78a-4e02-a3e9-7a258c22da03 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] Loading remote schema: domain resolve_remote /usr/lib/python2.7/site-packages/designate/schema/resolvers.py:3811:08
castlemilk2017-04-27 21:05:35.495 28562 INFO eventlet.wsgi [req-7225b1fe-b78a-4e02-a3e9-7a258c22da03 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] 127.0.0.1 - - [27/Apr/2017 21:05:35] "GET /v1/domains HTTP/1.1" 200 882 0.70231111:08
castlemilk2017-04-27 21:05:35.506 28562 DEBUG designate.api.middleware [req-e41791c5-6a7e-4bfb-8d7b-00bcd01ddc6a 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] -------------CUSTOM_LOGGING---------------- make_context /usr/lib/python2.7/site-packages/designate/api/middleware.py:12811:08
castlemilk2017-04-27 21:05:35.507 28562 DEBUG designate.api.middleware [req-e41791c5-6a7e-4bfb-8d7b-00bcd01ddc6a 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] <designate.context.DesignateContext object at 0x61941d0> make_context /usr/lib/python2.7/site-packages/designate/api/middleware.py:12911:08
castlemilk2017-04-27 21:05:35.509 28562 DEBUG designate.central.rpcapi [req-e41791c5-6a7e-4bfb-8d7b-00bcd01ddc6a 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] Calling designate.central.find_zone() over RPC wrapped /usr/lib/python2.7/site-packages/designate/loggingutils.py:2411:08
castlemilk2017-04-27 21:05:35.531 28562 DEBUG designate.central.rpcapi [req-e41791c5-6a7e-4bfb-8d7b-00bcd01ddc6a 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] Calling designate.central.find_record() over RPC wrapped /usr/lib/python2.7/site-packages/designate/loggingutils.py:2411:08
castlemilk2017-04-27 21:05:35.550 28562 INFO eventlet.wsgi [req-e41791c5-6a7e-4bfb-8d7b-00bcd01ddc6a 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] 127.0.0.1 - - [27/Apr/2017 21:05:35] "DELETE /v1/domains/055b9a01-0d8f-49af-b53c-3eb733e1601a/records/03947911-4005-4f95-8bc1-a28aa9d41ce0 HTTP/1.1" 400 351 0.04703011:08
castlemilkthis is the code added:11:08
castlemilkLOG.debug("-------------CUSTOM_LOGGING----------------")11:08
castlemilk    129             LOG.debug(ctxt)11:08
castlemilklet me know if u want something more11:08
castlemilkbrb11:08
castlemilkand incase u miss it, this is the version:11:10
castlemilkdesignate-3.0.0-py2.7.egg-info11:10
mugsiecastlemilk: hey, sorry, back11:22
mugsiecan you update the lines to ctxt.to_dict() and request.to_dict()11:22
mugsiealso, paste.openstack.org is better for pasting large blocks of text11:23
*** MarkBaker has quit IRC11:37
*** MarkBaker has joined #openstack-dns11:39
castlemilkok ill do thata now11:42
*** vlad_ has joined #openstack-dns11:45
castlemilkok heres the logs http://paste.openstack.org/show/608197/11:46
castlemilkmugsie: http://paste.openstack.org/show/608197/11:47
castlemilk@mugsie: http://paste.openstack.org/show/608197/11:47
mugsiecrap, that look like it is OK11:47
mugsieOK - can you add a log of context.to_dict() underneath https://github.com/openstack/designate/blob/ca615479a3ab624e391baf483004e72f102e0202/designate/central/service.py#L1706 ?11:49
mugsieand a log of context.edit_managed_records11:49
castlemilkyep11:53
castlemilkhmm im looking in the central.log and my log messages arent showing11:59
castlemilkif recordset.managed and not context.edit_managed_records:12:00
castlemilk            LOG.debug("-------CUSTOM_LOG---------------2\n")12:00
castlemilk            LOG.debug(context.to_dict())12:00
castlemilk            LOG.debug("-------CUSTOM_LOG---------------3\n")12:00
castlemilk            LOG.debug(context.edit_managed_records)12:00
castlemilk            raise exceptions.BadRequest('Managed records may not be deleted')12:00
mugsielooks good12:00
castlemilkthats what i've added12:00
mugsieand you are running debug logging?12:00
castlemilk2017-04-27 21:59:15.595 5489 DEBUG designate.storage.impl_sqlalchemy [req-ff757a42-c70d-467d-9df2-f04a4206211f 914e9e9b0e28430bbf5c0641374a9a9c f8e9ed2aaf4c4d44a6b74cdde3faf75d - - -] Fetched zone <Zone id:'055b9a01-0d8f-49af-b53c-3eb733e1601a' type:'PRIMARY' name:'mgmt.pants.net.' pool_id:'794ccc2c-d751-44fe-b57f-12:00
castlemilki see that in the log12:00
castlemilkso i presume so12:00
castlemilklet me confirm12:00
castlemilk########################12:01
castlemilk## General Configuration12:01
castlemilk########################12:01
castlemilk# Show more verbose log output (sets INFO log level output)12:01
castlemilkverbose = True12:01
castlemilk# Show debugging output in logs (sets DEBUG log level output)12:01
castlemilkdebug = True12:01
castlemilkso i guess that if condition isnt being hit?12:01
castlemilklet me try put it outside12:01
mugsieah, there are 2 places that this is checked for12:03
mugsieit might be hitting the other one12:03
*** MarkBaker has quit IRC12:04
castlemilkok ive added at the other spot12:05
castlemilkah looks like theres 4 spots12:08
castlemilkhmm cant seem to get to generate the custom logging in central.log12:11
castlemilki put it everywhere the exception is raised12:11
mugsie:/12:12
castlemilkis there a chance it doesnt make it there?12:12
odyssey4memugsie timsim is there any reason to hold back on merging https://review.openstack.org/445069 ?12:12
mugsieis there more than one central running?12:12
mugsieodyssey4me: nope12:12
castlemilkps -ax | grep central12:13
castlemilk 3026 pts/0    S+     0:00 grep --color=auto central12:13
castlemilk22598 ?        Ss     0:02 ceilometer-polling: master process [/usr/bin/ceilometer-polling --polling-namespaces central --logfile /var/log/ceilometer/central.log]12:13
castlemilk30632 ?        Ss     0:06 /usr/bin/python /usr/bin/designate-central --config-file /etc/designate/designate.conf --log-file /var/log/designate/central.log12:13
castlemilkonly one12:13
castlemilkon all control nodes12:13
odyssey4mealso, https://review.openstack.org/460404 LGTM and brings the tox config back to normal so if you have a gap to look through it it'd be appreciated12:13
mugsieI saw it, was look through it already12:13
mugsieWe still have eventlet 0.20 in that patch, but we can keep itterating12:14
odyssey4meyeah, with each iteration the problem is getting smaller12:14
odyssey4meor at least easier to break down into smaller pieces12:15
mugsieyeah12:15
mugsiek, thats merging now12:17
mugsiecastlemilk: could it be running on another machine?12:18
castlemilkno12:19
castlemilki have designate running on only one of the three control nodes12:19
*** EricGonc_ has quit IRC12:19
mugsieeh12:22
mugsie:/12:22
castlemilkim trying the logging outside the if12:22
mugsieah, crap12:23
mugsiemy bad12:23
mugsiehttps://github.com/openstack/designate/blob/master/designate/api/v1/records.py#L26912:23
castlemilkah lol ok12:23
mugsiewe do not allow deletion in the v1 API12:23
mugsiebut we do in v212:23
mugsiewhy the hell did that get in there12:24
* mugsie cannot wait to delete v1 forever12:24
castlemilkok so i need to use the V2 API?12:24
mugsieyeah12:24
castlemilkhow do i use that on the 2.6.0 client?12:24
mugsie2.6.0 should work12:24
mugsieits the openstack <command> cli12:25
mugsie2.3.0 has issues12:25
*** zhurong has quit IRC12:25
castlemilkyea but openstackclient is giving me the bool error12:25
castlemilkthe same one12:25
castlemilkas 2.3.012:25
mugsieoh, really?12:25
castlemilk[root@controller1 designate]# openstack recordset delete mgmt.pants.net. 4bfe6874-71e4-4294-9aa8-197d17f39641 --edit-managed12:26
castlemilkUnexpected exception for http://127.0.0.1:9001/v2/zones?name=mgmt.pants.net.: Header value True must be of type str or bytes, not <type 'bool'>12:26
*** MarkBaker has joined #openstack-dns12:26
castlemilk[root@controller1 designate]# openstack --version12:26
castlemilkopenstack 3.8.012:26
mugsietry --edit-managed True ?12:26
castlemilk[root@controller1 designate]# openstack recordset delete mgmt.pants.net. 4bfe6874-71e4-4294-9aa8-197d17f39641 --edit-managed True12:27
castlemilkusage: openstack recordset delete [-h] [-f {json,shell,table,value,yaml}]12:27
castlemilk                                  [-c COLUMN] [--max-width <integer>]12:27
castlemilk                                  [--print-empty] [--noindent]12:27
castlemilk                                  [--prefix PREFIX] [--all-projects]12:27
castlemilk                                  [--edit-managed]12:27
castlemilk                                  [--sudo-project-id SUDO_PROJECT_ID]12:27
castlemilk                                  zone_id id12:27
castlemilkopenstack recordset delete: error: unrecognized arguments: True12:27
castlemilklike that?12:27
mugsietry it just after the delete12:27
castlemilkroot@controller1 designate]# openstack recordset delete --edit-managed True mgmt.pants.net. 4bfe6874-71e4-4294-9aa8-197d17f3964112:28
castlemilkusage: openstack recordset delete [-h] [-f {json,shell,table,value,yaml}]12:28
castlemilk                                  [-c COLUMN] [--max-width <integer>]12:28
castlemilk                                  [--print-empty] [--noindent]12:28
castlemilk                                  [--prefix PREFIX] [--all-projects]12:28
castlemilk                                  [--edit-managed]12:28
castlemilk                                  [--sudo-project-id SUDO_PROJECT_ID]12:28
castlemilk                                  zone_id id12:28
castlemilkopenstack recordset delete: error: unrecognized arguments: 4bfe6874-71e4-4294-9aa8-197d17f3964112:28
mugsiegah12:28
castlemilkalso tried after delete without True12:28
kiallhttps://github.com/openstack/python-designateclient/commit/9ceb24149110662d65469f0554036f7fe3e3d2c3 <-- fixed that a while back?12:28
mugsiekiall: aparetnyl not12:29
castlemilkok looks like it works on 3.9.012:29
castlemilkof the openstack client12:29
mugsie-_-12:29
castlemilkbut not 3.8.012:30
*** fandi has quit IRC12:30
mugsiehuh12:30
mugsieweird12:30
castlemilkanyway thanks a lot!12:31
mugsieno porblem12:35
*** zhurong has joined #openstack-dns12:58
*** catintheroof has joined #openstack-dns13:17
*** stanzgy has quit IRC13:19
*** egonzalez has quit IRC13:47
*** kiall has quit IRC13:47
*** vlad_ has quit IRC13:50
openstackgerritMerged openstack/designate master: Refactor NSD4 UT for using upstream eventlet  https://review.openstack.org/46040413:53
*** kiall has joined #openstack-dns13:53
*** yee379 has quit IRC13:55
*** yee379 has joined #openstack-dns13:56
*** EricGonczer_ has joined #openstack-dns14:03
*** mlavalle has joined #openstack-dns14:04
*** kiall has quit IRC14:23
*** zhurong has quit IRC14:39
*** kiall has joined #openstack-dns14:41
*** MarkBaker has quit IRC14:54
*** pcaruana has quit IRC15:05
*** MarkBaker has joined #openstack-dns15:06
*** zigo has quit IRC15:17
openstackgerritMerged openstack/designate master: Updated from global requirements  https://review.openstack.org/44506915:36
*** kiall has quit IRC15:38
*** kiall has joined #openstack-dns15:38
*** kiall has joined #openstack-dns15:38
*** castlemi_ has joined #openstack-dns15:46
*** castlemilk has quit IRC15:46
*** egonzalez has joined #openstack-dns15:50
*** egonzalez has quit IRC16:02
*** egonzalez has joined #openstack-dns16:05
*** egonzalez has quit IRC16:06
*** chlong_ has joined #openstack-dns16:08
*** chlong_ has quit IRC16:14
*** abalutoiu has quit IRC16:26
*** abalutoiu has joined #openstack-dns17:16
*** MarkBaker has quit IRC17:21
*** MarkBaker has joined #openstack-dns18:07
*** amoralej is now known as amoralej|off18:16
openstackgerritDiana Clarke proposed openstack/designate master: Enable posargs for the py35 tests  https://review.openstack.org/45045118:24
*** carthaca_ has quit IRC18:36
*** david_4 has quit IRC18:36
*** sapcc-bot has quit IRC18:36
*** david_5 has joined #openstack-dns18:36
*** sapcc-bot2 has joined #openstack-dns18:36
*** carthaca_2 has joined #openstack-dns18:36
*** Daviey_ has joined #openstack-dns18:38
*** Daviey has quit IRC18:38
*** odyssey4me has quit IRC18:38
*** dhellmann has quit IRC18:38
*** odyssey4me_ has joined #openstack-dns18:39
*** dhellmann has joined #openstack-dns18:39
*** openstackgerrit has quit IRC18:48
*** MarkBaker has quit IRC19:07
*** egonzalez has joined #openstack-dns19:54
*** MarkBaker has joined #openstack-dns19:55
*** openstackgerrit has joined #openstack-dns19:57
*** ChanServ sets mode: +v openstackgerrit19:57
openstackgerritDiana Clarke proposed openstack/designate master: Unskip py3 tests that no longer fail  https://review.openstack.org/46072719:57
openstackgerritDiana Clarke proposed openstack/designate master: Unskip py3 tests that no longer fail  https://review.openstack.org/46072720:13
*** EricGonczer_ has quit IRC20:44
*** EricGonczer_ has joined #openstack-dns20:45
*** castlemi_ has quit IRC21:09
*** EricGonc_ has joined #openstack-dns21:28
*** EricGonczer_ has quit IRC21:29
*** castlemilk has joined #openstack-dns21:30
*** zigo has joined #openstack-dns21:32
*** kbyrne has quit IRC21:50
*** kbyrne has joined #openstack-dns21:54
*** catintheroof has quit IRC22:00
*** castlemilk has quit IRC22:03
*** timsim has quit IRC22:17
*** filler has quit IRC22:17
*** filler has joined #openstack-dns22:20
*** castlemilk has joined #openstack-dns22:20
*** timsim has joined #openstack-dns22:26
*** castlemilk has quit IRC22:32
*** castlemilk has joined #openstack-dns22:58
*** EricGonc_ has quit IRC23:05
*** mlavalle has quit IRC23:42

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!