Wednesday, 2014-09-10

*** rmoe has quit IRC00:14
*** eankutse has joined #openstack-dns00:14
*** eankutse has quit IRC00:17
*** eankutse has joined #openstack-dns00:17
*** darshan104 has quit IRC00:30
*** rmoe has joined #openstack-dns00:35
*** russellb has quit IRC00:46
*** russellb has joined #openstack-dns00:49
*** nosnos has joined #openstack-dns01:54
*** amcrn has quit IRC01:55
*** eankutse1 has joined #openstack-dns02:23
*** eankutse has quit IRC02:26
*** eankutse1 has quit IRC02:27
*** richm has quit IRC02:29
*** k4n0 has joined #openstack-dns05:45
*** darshan104 has joined #openstack-dns06:09
*** k4n0 has quit IRC07:07
*** k4n0 has joined #openstack-dns07:22
openstackgerritOpenStack Proposal Bot proposed a change to openstack/designate: Updated from global requirements  https://review.openstack.org/12033107:33
*** darshan104 has quit IRC08:10
*** nosnos has quit IRC09:41
*** nosnos has joined #openstack-dns09:42
*** nosnos has quit IRC09:47
openstackgerritJordan Pittier proposed a change to openstack/designate: Devstack setup : use get_or_create_* helpers for user and endpoint creation  https://review.openstack.org/12036310:02
openstackgerritA change was merged to openstack/designate: Add index on powerdns.records.designate_record_id  https://review.openstack.org/12008610:53
openstackgerritJordan Pittier proposed a change to openstack/designate: Devstack setup : use get_or_create_* helpers  https://review.openstack.org/12036311:19
*** ekarlso- has quit IRC11:37
*** ekarlso- has joined #openstack-dns11:37
*** mwagner_lap has quit IRC11:38
openstackgerritA change was merged to openstack/designate: Updated from global requirements  https://review.openstack.org/12033112:06
openstackgerritJordan Pittier proposed a change to openstack/designate: Devstack Integration : Fix Bashate E20  https://review.openstack.org/12039912:21
*** mwagner_lap has joined #openstack-dns12:37
*** mwagner_lap has quit IRC12:41
*** mwagner_lap has joined #openstack-dns12:41
*** jordanP has joined #openstack-dns12:43
KialljordanP: I'm guessing you're the Jordan Pittier submitting some devstack fixes?12:56
jordanPKiall, that's me sir :)12:56
KiallCool - Thanks :)12:57
jordanPthose are not real issues but I saw somewhere that designate could be integrated into devstack soon12:57
KiallI'm guessing you're more familiar with devstack than I, do you happen to know if we have any other issues etc we should fix before we try and have it merged?12:57
jordanPyeah, there the use of local variable too12:57
jordanPthey should be lower case12:58
jordanPdean troyer made a cleanup series of patch recently, it will be nice to follow the "convention", though there's not a bashate check for this yet12:58
jordanPI mean, in lib/designate all local variables should be in lower case12:59
KiallSo - Style stuff, that's almost always an easy fix :) Glad there's no bigger issues (that we know of ;))12:59
jordanPyeah, Ive been playing with designate in a devstack, so far so good12:59
KiallCool :)13:01
*** richm1 has joined #openstack-dns13:07
openstackgerritA change was merged to openstack/designate: Devstack setup : use get_or_create_* helpers  https://review.openstack.org/12036313:10
*** nkinder has quit IRC13:13
*** eankutse has joined #openstack-dns13:13
*** eankutse has quit IRC13:18
*** eankutse has joined #openstack-dns13:18
*** eankutse has quit IRC13:25
openstackgerritKiall Mac Innes proposed a change to openstack/designate: Backends need to implement c/u/d_recordset  https://review.openstack.org/11976813:26
*** jmcbride has joined #openstack-dns13:32
*** rmoe has quit IRC13:35
*** rmoe has joined #openstack-dns13:37
openstackgerritJordan Pittier proposed a change to openstack/designate: Devstack Integration : Coding style fix  https://review.openstack.org/12039913:38
*** eankutse has joined #openstack-dns13:45
*** jmcbride has quit IRC13:47
*** jmcbride has joined #openstack-dns13:58
*** timsim has joined #openstack-dns14:00
*** nkinder has joined #openstack-dns14:03
*** eankutse has quit IRC14:04
*** eankutse has joined #openstack-dns14:07
*** zain has joined #openstack-dns14:32
*** zain has joined #openstack-dns14:32
*** EricGonczer_ has joined #openstack-dns14:33
*** paul_glass has joined #openstack-dns14:49
jordanPmugsie, my patch 'Devstack Integration : Coding style fix' failed, sorry about that, i missed something14:49
openstackgerritJordan Pittier proposed a change to openstack/designate: Devstack Integration : Coding style fix  https://review.openstack.org/12039914:56
*** EricGonczer_ has quit IRC14:58
*** EricGonczer_ has joined #openstack-dns14:59
*** EricGonczer_ has quit IRC15:01
*** jmcbride has quit IRC15:01
*** EricGonczer_ has joined #openstack-dns15:01
*** EricGonc_ has joined #openstack-dns15:02
*** eankutse has quit IRC15:06
*** eankutse1 has joined #openstack-dns15:06
*** EricGonczer_ has quit IRC15:06
*** paul_glass1 has joined #openstack-dns15:08
*** paul_glass has quit IRC15:11
*** vinod1 has joined #openstack-dns15:28
*** jmcbride has joined #openstack-dns15:32
*** k4n0 has quit IRC15:34
*** jmcbride has quit IRC15:40
*** diga has joined #openstack-dns15:42
*** eankutse1 has quit IRC15:58
jordanPguys, I can't find a designate client for api v2, is that normal ?15:59
*** eankutse has joined #openstack-dns15:59
vinod1jordanP - yes currently the designate client only supports api v116:00
jordanPvinod1, is there ongoing work to provide a client for api v2 ?16:00
timsimThere was some work done, but it never merged: https://review.openstack.org/#/q/project:openstack/designateclient,n,z16:00
jordanPcause I love the abilitity to set a ptrrecord on a floating ip16:00
jordanPtimsim, you meant https://review.openstack.org/#/q/project:openstack/python-designateclient,n,z :)16:02
timsimFor some reason this link did not copy https://review.openstack.org/#/c/76457/16:02
jordanPtimsim, do you know why https://review.openstack.org/#/c/76457/ was abandonned ?16:03
vinod1ekarlso- might know better16:04
jordanPi have to leave, i'll ask tomorrow16:07
*** jmcbride has joined #openstack-dns16:07
*** jordanP has quit IRC16:08
*** mwagner_lap has quit IRC16:09
*** rmoe has quit IRC16:13
Kiallvinod1: it was because the v2 api wasn't final :)16:22
vinod1that is what i assumed - but I wasn't sure - jordanP the originator of the question isn't around now16:24
ekarlso-Kiall: regarding that...16:24
ekarlso-I got branches locally ^16:24
*** rmoe has joined #openstack-dns16:30
*** jmcbride has quit IRC16:51
*** rjrjr has joined #openstack-dns16:57
*** rossk has joined #openstack-dns17:05
*** amcrn has joined #openstack-dns17:05
*** richm1 has quit IRC17:20
*** mwagner_lap has joined #openstack-dns17:33
*** richm1 has joined #openstack-dns17:34
vinod1rjrjr: I did read the specs17:37
rjrjri have a 'status' in the database table for pool manager, but i only need that for create/delete of a zone.17:37
rjrjrthe way i envision pool manager working for an update zone is:17:38
rjrjrloop through servers and call mini-dns zone change method17:39
rjrjrloop through servers and call poll for serial number17:39
rjrjrmini-dns' poll for serial number method will only report back to pool manager when it is successful (need context, domain, and serial number)17:40
rjrjrupdate_status (the call that mini-dns makes) updates the database and checks if it is has enough info to call update_status in central17:40
rjrjrit will base that call to central on whether or not the threshold number of servers has a new (i.e. higher) serial number than what the Designate database knows about17:41
rjrjrthen the update_status in central updates the Designate records table and changes pendings to active or active to deleted based on serial number17:42
vinod1so would the status be just pending  and never move to error?17:44
vinod1what if the threshold is 100% and one of the name servers is down?17:44
rjrjri don't know how you would move an update of an individual record to an error state.17:44
rjrjrwhat we could do is look at the pool manager's status table and report an error if a server is lagging the others or some other way of getting an error.17:46
vinod1that is what i was thinking17:46
rjrjrbut we can't base it on a particular call the user makes, otherwise we would need to track the through pool manager, which would balloon the pool manager status table.17:47
vinod1if we track the time in the pool manager status table too then periodically a thread could see something was pending for a long time - then send it as error to central17:47
rjrjrthat is the only way i see that working without impacting performance negatively.17:48
rjrjrbut figuring out which records are in 'Error' is going to be tricky.17:48
rjrjrdo you mark all records with a serial number or higher as Error?  or just some of them?17:49
vinod1once pool manager calls central's update_status with a serial number then17:49
vinod1central would mark all pending recordsets/domain changes upto that serial as error17:49
rjrjrremember, we are recording the 'desired' serial number when a request to update the zone comes in.17:50
rjrjrhow do we know the server is down versus just being slow?17:50
rjrjrdoes it even matter?17:50
vinod1the time should be configurable based on the admin's preferences17:51
rjrjri can do that.17:51
rjrjri'll add this.17:51
rjrjragain, we don't need mini-dns to report the status to pool manager.  i don't even need to know about the error'd requests.17:52
rjrjrthat will help cut down on calls between mini-dns and pool manager.17:52
rjrjr(and hence calls to the pool manager table.)17:52
vinod1how will pool manager know when a particular change is on the nameserver?17:53
rjrjrmini-dns is sending successful polls back to pool manager.17:53
rjrjrwhich should be most polls anyway (unless there is a problem.)17:54
rjrjrwhich we'll report as you suggested.17:54
vinod1Essentially you are saying that minidns will call update_status only on a success?17:55
rjrjryes.17:55
rjrjrwe are giving minidns X amount of time to get a success.  (defaults to about 1.5 minutes.)17:55
vinod1the minidns has retries and expire based on the number of retries - whereas the pool manager would mark something as error based on time17:55
rjrjrtime or attempts or number of serial numbers we are behind or ...17:56
rjrjrwe can base it on any number of things.17:56
vinod1so if minidns has 3 retries and it takes 3 minutes - minidns could still be retrying when pool manager marks something as error17:56
rjrjryes.17:57
vinod1so we need to handle the case when minidns reports a success for something that the pool manager already marked as an error17:57
rjrjrthat is built in already.17:58
vinod1Ideally if pool manager is basing everything on time, then minidns too should do the same17:58
rjrjrvinod: think of the case where someone wants to add 100 records and writes a program or script to do that.  think of how many calls we are talking about if we had 10 servers in 1 pool.17:58
rjrjrDesignate doesn't have a 'batch' call to add all 100 at once.17:59
rjrjrwe are looking at 1000 calls to mini-dns being fired off pretty quickly.17:59
vinod1we did talk about adding 'bulk' calls being added to designate18:00
rjrjrsure, but we don't have it right now.  and it doesn't stop a user from doing it one record at a time even if we did.18:00
vinod1in this particular case, we could have the pool manager do an optimization - delay sending requests to minidns for sometime, if an update comes in between then repeat the process18:01
rjrjran atomic call is going to be needed if we want to support nsupdate (dynamic DNS) regardless.18:01
rjrjrthat is too much.18:01
rjrjror we just make sure the calls between pool manager and mini-dns are pertinent and only for successes.18:01
*** diga has quit IRC18:01
rjrjrthat is how i designated the pool manage update_status call.18:02
vinod1How many errors would we normally see - would that save much - or would it just complicate the design?18:03
rjrjrit saves a call and talking to the database.18:03
rjrjri'll just ignore errors regardless in pool manager. 8^)18:04
rjrjrIMO, chatting on rabbitmq should be kept minimal and calling the database from pool manager should be minimum.18:04
rjrjrthe design is simpler by not passing a status.18:05
vinod1if you don't update the database, on error, how would the pool manager keep track of errors later if minidns reports a success?18:05
rjrjri'm not tracking errors, i'm tracking serial numbers.18:05
rjrjrerrors shouldn't matter.18:05
rjrjrin order for me to track error, i would need to know for what, then i'm back to having a request ID and the database grows exponentially.18:06
vinod1i think we are optimizing prematurely even before we have something working correctly18:06
openstackgerritDoug Hellmann proposed a change to openstack/designate-specs: Add RSS feed  https://review.openstack.org/12050118:07
rjrjrvinod, one of the things that was made clear to me in the offsite is the pool manager database cannot have a row for every request.18:07
openstackgerritDoug Hellmann proposed a change to openstack/designate-specs: Remove templates from toctrees  https://review.openstack.org/12050218:07
openstackgerritDoug Hellmann proposed a change to openstack/designate-specs: Use the current date for the copyright statement  https://review.openstack.org/12050318:07
rjrjrwe have a row for every server - zone update18:07
rjrjrpotentially a row for every server zone create18:08
rjrjrpotentially a row for every server - zone delete18:08
rjrjrif i have 10 servers in 1 pool that is 30 rows potentially18:08
rjrjrtrust me when i'm telling you, i followed KISS when i did this design.18:09
rjrjri kept it as simple as i possibly could.18:09
rjrjroptimization wasn't why it turned out this way, it was to keep it simple.18:10
vinod1did we talk about having the pool_manager_status in memcache?18:11
rjrjrwe did.  everyone wanted to try database first.  it doesn't fix the issue either.18:11
rjrjrif i track a single request through pool manager, we have to track all 10 servers.  what if one is down.  do i hold onto those requests individually for hours or days until that server is back online?18:12
rjrjror do i remove them when the threshold is met?  how do i know when a call to update_status is for a request that has "already been handled" and removed or for a new request?  you can see how complicated this gets.18:13
rjrjrit seems to me what is important is the zone and the serial number.18:14
rjrjrso, the status table stores the server - zone - serial number18:14
rjrjr(for updates)18:14
rjrjrone row regardless of how many requests.  i just keep updating the serial number and once the lowest serial number for a threshold increases, that is time to call central's update_status.18:15
rjrjrsimple and elegant.18:15
rjrjri do store status for create and delete in the same table.18:16
*** vinod1 has quit IRC18:21
*** vinod1 has joined #openstack-dns19:11
*** mgagne has quit IRC19:18
*** mgagne has joined #openstack-dns19:19
*** mgagne is now known as Guest7525019:19
*** rossk has quit IRC19:25
openstackgerritDoug Hellmann proposed a change to openstack/designate-specs: Remove templates from toctrees  https://review.openstack.org/12050219:40
openstackgerritDoug Hellmann proposed a change to openstack/designate-specs: Use the current date for the copyright statement  https://review.openstack.org/12050319:40
*** sballe has joined #openstack-dns19:52
*** vinod1 has quit IRC19:59
*** tmc3inphilly has joined #openstack-dns20:00
*** tmc3inphilly has left #openstack-dns20:00
openstackgerritDoug Hellmann proposed a change to openstack/designate-specs: Remove templates from toctrees  https://review.openstack.org/12050220:01
openstackgerritDoug Hellmann proposed a change to openstack/designate-specs: Use the current date for the copyright statement  https://review.openstack.org/12050320:01
openstackgerritDoug Hellmann proposed a change to openstack/designate-specs: Add RSS feed  https://review.openstack.org/12050120:01
*** vinod1 has joined #openstack-dns20:06
*** jmcbride has joined #openstack-dns20:14
*** vinod1 has quit IRC20:18
*** vinod1 has joined #openstack-dns20:20
*** jmcbride has quit IRC20:21
*** vinod1 has quit IRC20:31
*** amcrn has quit IRC20:35
*** vinod1 has joined #openstack-dns20:44
*** jmcbride has joined #openstack-dns21:05
*** paul_glass has joined #openstack-dns21:11
*** paul_glass1 has quit IRC21:11
*** vinod1 has quit IRC21:17
*** paul_glass has quit IRC21:18
*** vinod1 has joined #openstack-dns21:19
*** EricGonc_ has quit IRC21:23
*** jmcbride has quit IRC21:27
*** jmcbride has joined #openstack-dns21:29
*** vinod1 has quit IRC21:31
*** vinod1 has joined #openstack-dns21:42
*** sballe_ has joined #openstack-dns21:46
*** timsim has quit IRC21:47
*** jmcbride has quit IRC21:49
*** sballe has quit IRC21:49
*** amcrn has joined #openstack-dns21:53
*** eankutse has quit IRC22:07
*** nkinder has quit IRC22:12
*** zain has quit IRC22:35
*** nkinder has joined #openstack-dns23:02
openstackgerritOpenStack Proposal Bot proposed a change to openstack/python-designateclient: Updated from global requirements  https://review.openstack.org/12060823:09
*** EricGonczer_ has joined #openstack-dns23:20
openstackgerritVinod Mangalpally proposed a change to openstack/designate: Getting deleted recordsets returns a 404  https://review.openstack.org/12061923:33
*** vinod1 has quit IRC23:34
*** richm1 has quit IRC23:44

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!