Monday, 2015-03-02

*** stanzgy has joined #openstack-dns01:34
*** stanzgy_ has joined #openstack-dns01:34
*** richm has quit IRC02:19
*** mwagner_lap has joined #openstack-dns02:35
*** GonZo2K has quit IRC03:28
*** nkinder has joined #openstack-dns03:38
*** csoukup has joined #openstack-dns03:55
*** andrewbogott_afk is now known as andrewbogott04:35
*** andrewbogott is now known as andrewbogott_afk04:41
*** csoukup has quit IRC04:47
openstackgerritOpenStack Proposal Bot proposed openstack/designate: Imported Translations from Transifex  https://review.openstack.org/16008406:35
*** chlong has quit IRC08:02
zigo_andrewbogott_afk: I think it's just best if you rebuild designate from Debian's sources.09:01
*** jordanP has joined #openstack-dns10:05
*** stanzgy has quit IRC10:53
*** stanzgy_ has quit IRC10:54
*** untriaged-bot has joined #openstack-dns11:02
untriaged-botUntriaged bugs so far:11:02
untriaged-bothttps://bugs.launchpad.net/designate/+bug/141633711:02
openstackLaunchpad bug 1416337 in Designate "Designate server create with concurrent request is not listing all servers even after successful creation." [Undecided,Incomplete]11:02
untriaged-bothttps://bugs.launchpad.net/designate/+bug/142566811:02
openstackLaunchpad bug 1425668 in Designate "Poor error message when using same database for designate and the pool manager cache" [Undecided,New]11:02
untriaged-bothttps://bugs.launchpad.net/designate/+bug/142511711:02
openstackLaunchpad bug 1425117 in Designate "Designate does not work with postgres" [High,New]11:02
*** untriaged-bot has quit IRC11:02
openstackgerritKiall Mac Innes proposed openstack/designate: Implement Scoped TSIG Key Support  https://review.openstack.org/16016511:59
openstackgerritKiall Mac Innes proposed openstack/designate: WIP: Service Class Cleanup - Part 3/3  https://review.openstack.org/16012912:06
*** GonZo2K has joined #openstack-dns12:58
*** GonZo2K has quit IRC12:58
*** GonZo2K has joined #openstack-dns12:58
*** andrewbogott_afk is now known as andrewbogott13:44
*** fouxm has quit IRC13:54
*** fouxm has joined #openstack-dns13:57
*** richm has joined #openstack-dns14:06
*** nkinder has quit IRC14:19
*** vinod1 has joined #openstack-dns14:40
*** csoukup has joined #openstack-dns15:03
*** vinod1 has quit IRC15:13
*** nkinder has joined #openstack-dns15:13
*** paul_glass has joined #openstack-dns15:17
openstackgerritMerged openstack/designate: Imported Translations from Transifex  https://review.openstack.org/16008415:37
*** paul_glass has quit IRC15:47
*** jmcbride has joined #openstack-dns15:47
*** vinod1 has joined #openstack-dns16:02
openstackgerritSatyanarayana Patibandla proposed openstack/python-designateclient: Added previllege to admin tenant to view other tenants created records.  https://review.openstack.org/16039716:21
*** boris-42 has quit IRC16:32
openstackgerritTim Simmons proposed openstack/designate: Agent configuration and BIND9 improvements  https://review.openstack.org/15846216:42
*** rmoe has quit IRC16:47
*** paul_glass has joined #openstack-dns16:50
*** jordanP has quit IRC17:02
*** untriaged-bot has joined #openstack-dns17:02
untriaged-botUntriaged bugs so far:17:02
untriaged-bothttps://bugs.launchpad.net/designate/+bug/141633717:02
openstackLaunchpad bug 1416337 in Designate "Designate server create with concurrent request is not listing all servers even after successful creation." [Undecided,Incomplete]17:02
untriaged-bothttps://bugs.launchpad.net/designate/+bug/142566817:02
openstackLaunchpad bug 1425668 in Designate "Poor error message when using same database for designate and the pool manager cache" [Undecided,New]17:02
untriaged-bothttps://bugs.launchpad.net/designate/+bug/142511717:02
openstackLaunchpad bug 1425117 in Designate "Designate does not work with postgres" [High,New]17:02
*** untriaged-bot has quit IRC17:02
*** jordanP has joined #openstack-dns17:03
*** mugsie has quit IRC17:05
*** mugsie has joined #openstack-dns17:06
*** mugsie is now known as Guest4600017:06
*** GonZo2K has quit IRC17:07
*** rmoe has joined #openstack-dns17:08
*** Guest46000 has quit IRC17:19
*** Guest46000 has joined #openstack-dns17:19
*** Guest46000 is now known as mugsie_17:19
Kialltimsim: about? re https://review.openstack.org/#/c/160127/ - I honestly forgot I removed the constructor "config" option from that one, I think that's the answer to the Q I had left here.. https://review.openstack.org/#/c/160127/2/designate/agent/service.py17:22
timsimYeah i'm here17:23
Kiallby making "config" part of the standard constructor args, we can just read them from there.. Down the line, I'd like to see the config options be defined DRY-ly .. But, for now I'm aiming to get DNSService used fby both mDNS and Agent, so I can do the TSIG parts of mDNS17:23
timsimYeah my only thought there was if it might make reloadable config harder17:24
KiallSpeaking of which - The agent and ACL - We don't currently restrict who can send commands in, right?17:24
timsimThere's an allow-notify list17:24
timsimThat checks every request's source IP. if it's not in the list it refuses it.17:24
KiallOkay.. So TSIG isn't needed there to identify the caller - which makes my life easier for now :)17:25
timsimYeah not yet, I was really hoping to add it at some point, as people will definitely want it. But not necessary for now.17:25
KiallAny other thoughts on the 3 parts of that change? I'll update the config stuff now..17:26
timsimNah I didn't have anything else. Looked pretty clean. I haven't tested it out though.17:27
KiallCool :)17:31
*** boris-42 has joined #openstack-dns18:05
andrewbogottzigo: did you say that you have Trusty/Icehouse packages for designate someplace?  Or just Jessie/Icehouse?  (Even the latter would move me forward quite a bit.)18:06
openstackgerritGraham Hayes proposed openstack/designate: Fixed _is_superdomain check in central  https://review.openstack.org/15869318:09
Kiallandrewbogott: https://packages.debian.org/search?keywords=designate&searchon=names&suite=unstable&section=all18:17
andrewbogottKiall: Would you expect those to work on Trusty as well?  I landed in dependency hell when I tried to build something similar.18:18
KiallI suspect dependancy hell can be solved with the icehouse ubuntu cloud archive, and rebuilding the designate package after modifying the debian/control file to match whatever the cloud archive has available.. the versions there should be OK, since we pin to the same versions of deps as the rest of openstack..18:19
Kiallso even if debian has a newer older, what's in the cloud archive should be good for our icehouse release18:20
Kiallnewer or older versions*18:20
andrewbogottThere is no icehouse cloud archive for Trusty though…18:20
andrewbogottBut, ok, rebuilds will probably get me there.  Thanks18:20
KiallOhh, for trusty.. that shipped with icehouse, right? If so, then the stock ubuntu archive should have what we need18:21
*** mugsie_ is now known as mugsie18:22
KiallAnyway! Dependancy wise - we are tied to the versions used by the rest of OpenStack Icehouse, so if nova etc work on it, then so will we (most likely anyway!)..  So, updating the debian/control to match what's available should do it.. The only exception would be if the packages use deb packaging features not available on Trusty .. But I'm not really sure if they do/don't etc etc18:24
*** jordanP has quit IRC18:25
openstackgerritKiall Mac Innes proposed openstack/designate: Service Class Cleanup - Part 2/3  https://review.openstack.org/16012818:25
openstackgerritKiall Mac Innes proposed openstack/designate: WIP: Service Class Cleanup - Part 3/3  https://review.openstack.org/16012918:25
openstackgerritKiall Mac Innes proposed openstack/designate: Service Class Cleanup - Part 1/3  https://review.openstack.org/16012718:25
Kialltimsim / mugsie.. updated re the service config discussion we had.. Using a mix of both your suggestions ;)18:25
andrewbogotthm… may be my packaging skills are just not up to the job; I will persevere18:25
KiallYea - It's been about a year since I touched debian packaging too ;)18:26
openstackgerritKiall Mac Innes proposed openstack/designate: Service Class Cleanup - Part 2/3  https://review.openstack.org/16012818:32
openstackgerritKiall Mac Innes proposed openstack/designate: WIP: Service Class Cleanup - Part 3/3  https://review.openstack.org/16012918:32
openstackgerritKiall Mac Innes proposed openstack/designate: Service Class Cleanup - Part 1/3  https://review.openstack.org/16012718:32
KiallAlways miss something in the first 3 attempts at a long chain ;)18:32
*** jmcbride has quit IRC18:42
*** jmcbride has joined #openstack-dns18:52
*** crc32 has joined #openstack-dns18:55
openstackgerritMerged openstack/designate: Add howto for creating PTR records  https://review.openstack.org/15799518:59
*** crc32 has quit IRC19:03
*** GonZo2K has joined #openstack-dns19:11
*** GonZo2K has quit IRC19:12
openstackgerritTim Simmons proposed openstack/designate: Agent configuration and BIND9 improvements  https://review.openstack.org/15846219:12
*** crc32 has joined #openstack-dns19:12
*** shakamunyi has joined #openstack-dns19:20
KiallGreat.. pep8 fail in commit 1 of 3.. heh :(19:23
* Kiall needs `git rebase` to automatically do a `tox -epy27,pep8` on every commit in the chain -_-19:23
openstackgerritKiall Mac Innes proposed openstack/designate: Service Class Cleanup - Part 2/3  https://review.openstack.org/16012819:24
openstackgerritKiall Mac Innes proposed openstack/designate: Service Class Cleanup - Part 3/3  https://review.openstack.org/16012919:24
openstackgerritKiall Mac Innes proposed openstack/designate: Service Class Cleanup - Part 1/3  https://review.openstack.org/16012719:24
*** GonZo2K has joined #openstack-dns19:26
*** GonZo2K has quit IRC19:26
*** GonZo2K has joined #openstack-dns19:26
elarsonrjrjr: I was wondering if you might be able to help me understand the locking https://github.com/openstack/designate/blob/master/designate/pool_manager/service.py#L235 I'm curious why update has a lock while the other operations don't19:39
elarsonrjrjr: I'm also wondering if the lock doesn't really protect against anything if you were running more than one pool manager process19:40
Kiallelarson: I remember a conversation about this at the mid-cycle, but I'm struggeling to remember the exact reasoning..19:44
KiallI think the reasoning was that, since N update_statuses can come in at once, that method needs be wrapped in a lock to work at all even with a single pool manager..  And that we'll need something more for reliablity working with multiple active pool managers per pool..19:46
* elarson is reading the code more closely :)19:49
*** crc32 has quit IRC19:56
elarsonKiall: it looks like, generally, getting a serial consensus across backends could be problematic. that is my best guess at the moment :)19:57
elarsonproblematic == non-trivial to get a single, canonical answer19:58
KiallRace condition wise? With a single pool manager, it's doable as our consensus is based on the question "Do you have at least serial #1234" rather than "Do you have exactly serial #1234"19:58
KiallWith multiple pool managers and no lock, things get more interesting with race conditions.. and it may be that we need to lock cross all the pool managers for a particular pool :/19:59
*** johnbelamaric has joined #openstack-dns20:07
*** johnbelamaric1 has joined #openstack-dns21:04
*** johnbelamaric has quit IRC21:05
*** shakamunyi has quit IRC21:07
*** shakamunyi has joined #openstack-dns21:08
timsimCan we just not upgrade eventlet?21:22
*** chlong has joined #openstack-dns21:24
Kialltimsim: lol, what?21:24
timsimI've got a deployment that's got the latest code, but works with eventlet 0.16.1 and doesn't w/ 0.17.121:25
KiallAh.. 0.17.0 broke dns.name.Name, and 0.17.1 exposed some issues with our code..21:26
timsimhttp://paste.openstack.org/show/185485/21:26
Kiall(The 2 gate blockers we had in 2 days ;))21:26
KiallOO21:26
KiallHumm - That's different21:26
timsimI mean I knew there were issues. Patched them, thought it was good, but just this notify exchange fails.21:27
KiallI've not noticed that, is it happening in the gate? or just in your env?21:27
timsimOh just my crazy env21:27
timsimIt's MDNS->Proxy->Agent Notify via TCP21:28
Kialllol21:28
KiallProxy?21:28
timsimLong story.21:28
KiallUDP Notify certainly working (Just saw one ;))21:28
timsimIt's HAProxy. Yeah they're TCP21:28
timsimAnd it works fine on old eventlet.21:29
KiallI suppose they could have broken something else we haven't noticed yet :/21:30
timsimThe odd thing is that my CREATE/DELETE messages go through fine.21:30
elarsons/eventlet/gevent and monkey patch everything?21:30
KiallI can tell you straight out  - s/eventlet/gevent/ will result in whatever DC you're running this in going on fire...21:32
elarsonno big deal, just grab a fire extinguisher! problem solved21:33
timsimIt seems like the only thing they're patching that we use now is dns.query (which is what's sending the NOTIFY) https://github.com/eventlet/eventlet/commit/3de49aef1399594b2cab0652d9ff3976d19b5e72#diff-47bd8cdaac88cccf21624af4df7a35c3R4721:34
elarsonKiall: btw, I'd be surprised if we need that locking since we're using an eventlet threadpool21:35
elarsonthat said, I would *not* be surprised if I were wrong ;)21:35
*** andrewbogott is now known as andrewbogott_afk21:41
*** andrewbogott_afk is now known as andrewbogott21:43
Kiallelarson: at points within the lock, we do I/O, which gives eventlet time to switch to another thread, and quite possibly reenter the same method for another status update..21:44
KiallSo close -_-21:50
Kiall2015-03-02 21:50:41.981 DEBUG designate.service [-] Handling UDP Request from: 127.0.0.1:39464 from (pid=32240) _dns_handle_udp /opt/stack/designate/designate/service.py:30721:50
Kiall2015-03-02 21:50:41.986 WARNING dns.tsig [-] Keyname: test-key.21:50
Kiall2015-03-02 21:50:41.987 WARNING dns.tsig [-] Secret: +Cdjlkef9ZTSeixERZ433Q==21:50
Kiall2015-03-02 21:50:41.988 ERROR designate.dnsutils [-] Invalid TSIG signature from 127.0.0.1:3946421:50
*** chlong has quit IRC21:51
*** shakamunyi_ has joined #openstack-dns21:53
*** shakamunyi has quit IRC21:54
openstackgerritMerged openstack/designate: Remove explicit thread-local update in context  https://review.openstack.org/16011421:59
*** nkinder has quit IRC22:01
openstackgerritMerged openstack/designate: Sync with oslo-incubator  https://review.openstack.org/16011522:07
*** johnbelamaric has joined #openstack-dns22:08
openstackgerritMerged openstack/designate: Update default managed_resource_email  https://review.openstack.org/16017122:09
openstackgerritMerged openstack/designate: Add Developer Guidelines docs page  https://review.openstack.org/16017422:09
*** johnbelamaric1 has quit IRC22:11
*** GonZoPT has joined #openstack-dns22:22
*** GonZo2K has quit IRC22:24
*** csoukup has quit IRC22:40
KiallWarning.. openstackgerrit is about the spam us ;)22:43
openstackgerritKiall Mac Innes proposed openstack/designate: Implement Scoped TSIG Key Support  https://review.openstack.org/16016522:43
openstackgerritKiall Mac Innes proposed openstack/designate: Service Class Cleanup - Part 2/3  https://review.openstack.org/16012822:43
openstackgerritKiall Mac Innes proposed openstack/designate: Service Class Cleanup - Part 3/3  https://review.openstack.org/16012922:43
openstackgerritKiall Mac Innes proposed openstack/designate: Service Class Cleanup - Part 1/3  https://review.openstack.org/16012722:43
openstackgerritKiall Mac Innes proposed openstack/designate: WIP: Implement TSIG Support in mDNS  https://review.openstack.org/16054322:43
paul_glassvinod1: what behavior should I expect with the noop pool manager cache?22:45
*** jmcbride has quit IRC22:45
vinod1It is mostly used with unit tests.22:46
vinod1Theoritically if you use it - it should fetch from the minidns for every access to the cache22:47
vinod1Though checking for consensus relies on cache entries - so that might not work22:48
vinod1rjrjr: Thoughts/comments on this?22:49
paul_glasswith noop cache, I see zones go to ERROR, which makes sense I guess.22:53
KiallIn theory, they shouldn't go to ERROR.. If they do, it's a bug22:56
Kialla cache can be wiped at any moment in time, and we should handle that when it happens.. The noop cache let's us simulate loosing the cache all the time.. ;)22:57
paul_glassbut if the pool manager doesn't do anything, there's no part of designate that knows if the zone is actually active on the nameserver, is there?22:59
paul_glasshow could the zone ver go to active?22:59
paul_glassever*22:59
*** johnbelamaric has quit IRC23:03
paul_glassoh, it's a noop cache, not a noop pool manager.23:04
*** nkinder has joined #openstack-dns23:05
*** shakamunyi_ has quit IRC23:06
Kiallvinod1 / timsim / mugsie - That chain of 5 patches above is about 98.2% of TSIG support in mDNS etc :)23:12
KiallStill some final bits to work out, but pretty much there23:12
vinod198.2% - you are really precise :-)23:13
Kiall;)23:13
mugsiewell, its missing the vital bit ;)23:13
Kiall:D23:14
rjrjrvinod1: the noop cache will use mdns, after the zone goes to ERROR.  what we talked about at the midcycle meetup was to have the sync recovery use mdns if the cache is unavailable.23:16
rjrjras it is, the noop cache works fine.  it just is slower than using an actual cache.23:17
vinod1Right now calculating the consensus, gets from mdns and tries to store in the caches to calculate the consensus23:17
Kiallrjrjr: ah, right.. it recovers it rather than prevents it going to ERROR right now.. makes sense23:17
paul_glassrjrjr: I'm seeing zones move to and remain in ERROR indefinitely with the noop cache (with either one or two nameservers)23:17
paul_glassas of latest master23:18
rjrjrthat is not how the code i wrote worked.23:18
rjrjrthe sync recovery will see it is not in cache and use mdns to get the status.23:19
rjrjrso, you call pool manager to update a zone using noop cache.  as kiall explains, there it will go to ERROR state every time because it is looking at cache.  but for recovery, if there is no cache, it does the correct thing.23:21
paul_glassrjrjr: I'm seeing these logs when the "Calling periodic_recovery" happens: http://paste.openstack.org/show/185582/23:21
openstackgerritKiall Mac Innes proposed openstack/designate: WIP: Implement TSIG Support in mDNS  https://review.openstack.org/16054323:21
Kiallpaul_glass: ahh - looks like the error isn't (directly) cache related:23:22
Kiall u"rndc: 'addzone' failed: already exists\n"23:22
KiallCould you have leftover zones in bind from a previous install etc of Designate maybe>23:22
KiallOr does every new zone show the same error message?23:23
paul_glassno, the add zone works the first time.23:23
rjrjrit looks like periodic_recovery is not checking mdns when it does not find the entry in cache.23:24
KiallOkay, and this only happens if you enable the noop cache? That should be easy enough to reproduce if it happens everytime..23:25
rjrjrwhat should happen is periodic_recovery will always check mdns anytime it needs to use cache and cache does not exist.  this includes determining whether or not to add the zone.23:25
paul_glassKiall: yeah, this is only with the noop cache.23:26
rjrjrjust a sec.  let me look at the patch i submitted.23:26
openstackgerritKiall Mac Innes proposed openstack/designate: Implement Scoped TSIG Key Support  https://review.openstack.org/16016523:26
openstackgerritKiall Mac Innes proposed openstack/designate: WIP: Implement TSIG Support in mDNS  https://review.openstack.org/16054323:26
openstackgerritKiall Mac Innes proposed openstack/designate: Service Class Cleanup - Part 2/3  https://review.openstack.org/16012823:26
KiallLast spam attack for tonight, I swear.23:26
openstackgerritKiall Mac Innes proposed openstack/designate: Service Class Cleanup - Part 3/3  https://review.openstack.org/16012923:26
openstackgerritKiall Mac Innes proposed openstack/designate: Service Class Cleanup - Part 1/3  https://review.openstack.org/16012723:26
KiallLast spam attack for tonight, I swear. ;)23:26
vinod1rjrjr: _retrieve_from_mdns retrieves from mdns and stores it in the cache - the storing in the cache fails for noop cache23:27
rjrjrvinod1: correct.  but for periodic_recovery if there is no cache, i don't use it.23:32
*** chlong has joined #openstack-dns23:33
rjrjrhttps://review.openstack.org/#/c/149428/11/designate/pool_manager/service.py23:34
rjrjrlines 310 - 34023:34
rjrjr_retrieve_status() is called with force_check set to True.  that means it gets the status from mdns.23:35
rjrjrif the domain exists (and it does in this case) then we log that the domain is already created.23:36
vinod1If there are multiple servers that need to be updated - how would consensus calculation work without any entries in the cache23:36
rjrjrthe same way.  we are passing force_check, so it always calls mdns in periodic_recovery if there is not status in cache.23:37
rjrjr_is_create_consensus() is called with force_check set to True.23:38
*** openstackgerrit has quit IRC23:38
*** openstackgerrit has joined #openstack-dns23:38
*** ChanServ sets mode: +v openstackgerrit23:38
rjrjri wasn't happy with force_check either, but the code did what we wanted it to per the mid cycle meetup.23:39
paul_glassI'm heading out, so I went ahead and submitted a bug for this: https://bugs.launchpad.net/designate/+bug/142742523:39
openstackLaunchpad bug 1427425 in Designate "Zones remain in ERROR indefinitely with the noop pool manager cache" [Undecided,New]23:39
*** paul_glass has quit IRC23:40
rjrjrhttps://review.openstack.org/#/c/149428/11/designate/pool_manager/service.py23:40
rjrjrlines 550 - 55923:40
rjrjr_retrieve_status() will get the status from cache first, if it doesn't exist, it gets it from mdns.23:41
openstackgerritGraham Hayes proposed openstack/designate: Create ObjectAdaptorMixin and API Mixins  https://review.openstack.org/16055223:41
openstackgerritGraham Hayes proposed openstack/designate: Move Zones to ObjectAdaptorMixin for API interactions  https://review.openstack.org/16055323:41
rjrjrit looks like you are using check_server the same way, except you don't have all the places that need it covered.23:43
vinod1I will fix that then. Thanks for looking into this rjrjr23:44
rjrjrjust as an aside, a _retrieve_from_cache() that calls mdns seems wrong.  that is why i had a _retrieve_status() which called _retrieve_from_cache() and _retrieve_from_mdns() as needed.23:45
rjrjrbut that might just be a personal preference.23:45
vinod1could you explain your reasoning?23:46
rjrjrthe method is called _retrieve_from_cache(), but then you have it retrieving it from mdns.23:46
vinod1ah - now i understand23:46
rjrjr_retrieve_status() doesn't say where you are getting the status.23:46
vinod1i will check if that would be an easy change to make23:47
*** andrewbogott is now known as andrewbogott_afk23:57
*** jmcbride has joined #openstack-dns23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!