Friday, 2015-03-27

*** jmcbride has joined #openstack-dns00:01
kfox1111ok. lets try this again... created a new domain. see the following in named's log: Mar 27 00:04:45 designate named[18015]: received control channel command 'addzone kfox-test3.cloud.pnnl.gov { type slave; masters { 127.0.0.1 port 5354;}; file "slave.kfox-test3.cloud.pnnl.gov....3d7758887"; };'00:04
kfox1111I then add a record, and: Mar 27 00:06:22 designate named[18015]: client 127.0.0.1#41417: received notify for zone 'kfox-test3.cloud.pnnl.gov': not authoritative00:05
kfox1111something's still wrong. :/00:06
kfox1111yeah. doing a dumpdb I see none of the designate stuff...00:08
*** km has joined #openstack-dns00:11
kfox1111ok. if I run this manually: rndc addzone 'kfox-test2.cloud.pnnl.gov { type slave; masters { 127.0.0.1 port 5354;}; file "slave.kfox-test2.cloud.pnnl.gov.af2ffbb5-c67b-4cc7-9497-fe6d938e9d87"; };' I get permission denied.00:16
kfox1111so thats probably part of it.00:16
kfox1111heh. chmod 770 /var/named00:24
kfox1111problem solved. :/00:24
*** kei_yama has joined #openstack-dns00:32
kfox1111not all there yet though... I added a new domain, it didn't try addzoneing. :/00:32
kfox1111and delete doesn't seem to work at all.00:32
kfox1111ah. there we go. bad rnd.conf.00:40
*** kfox1111 has quit IRC00:49
*** andrewbogott_afk is now known as andrewbogott00:54
*** andrewbogott is now known as andrewbogott_afk00:55
*** jmcbride has quit IRC01:02
*** crc32 has quit IRC01:16
*** stanzgy has joined #openstack-dns01:38
*** penick has quit IRC01:53
*** dank_ has quit IRC01:55
*** nkinder has joined #openstack-dns02:13
*** jmcbride has joined #openstack-dns02:51
stanzgyanyone alive?03:12
*** jmcbride has quit IRC04:02
*** MentalRay has joined #openstack-dns04:40
*** GonZo2K has quit IRC06:23
*** MentalRay has quit IRC06:40
*** chlong has quit IRC07:16
ekarlsostanzgy: yes :P08:51
ekarlsoin between :D08:51
*** jordanP has joined #openstack-dns08:53
stanzgyekarlso: hi ;D09:09
stanzgyI have noticed that https://review.openstack.org/#/c/167268/6 ends up with set "max_size=65535" on dns.message.to_wire()09:10
stanzgySo the AXFR larger than 65535 will still raise TooBig?09:11
ekarlsostanzgy: yeah, I think Kiall is working on it09:16
stanzgyekarlso: oh, I think we could split one large AXFR message into multiple small ones. I'm also making some attempts on this too :P09:20
stanzgyekarlso: and in Designate it seems ppl prefer raw sql query and where stmt rather than SQLAlchemy utilities like filter/relationship/etc.. Do you know the reason?09:23
openstackgerritEndre Karlson proposed openstack/designate: Add code to allow triggering of AXFR from API  https://review.openstack.org/16533110:12
Kiallstanzgy: yea, fixing greater than 65k is on the TODO.. Need to rework things slightly..10:14
KiallFor now, it getting to 65k was so trivial it made sense to just merge.10:14
Kiallre SQLAlchemy, many projects use the SQLA ORM, we don't.. many projects are wanting to switch away, but have too much code/momentum to do so, we managed it ;)10:15
KiallAnd, because of the distributed architecture, many of the SQLA ORM advantages just disapear.. e.g. lazy joinedloads don't work when the result has been passed over RPC10:17
stanzgyKiall: oh, the 65k corresponding laundpad bug has been marked as 'Fixed committed'. So I just thought no more further work on it. nevermind -.-10:19
KiallWhoops :)10:19
KiallRe-opened10:19
stanzgyKiall: agree with you on this point. but for now we have no database session and transaction mechanism which I think is very useful .10:21
KiallWe do actually use the SQLA Sesstions, and transactions ;)10:27
Kialle.g. here's one place we use it - https://git.openstack.org/cgit/openstack/designate/tree/designate/central/service.py#n11210:28
Kiallalong with the various methods decorated with @transaction inthere10:28
stanzgyI will re-test the #1434479 someday next week to see if our recently merged patches works, after https://review.openstack.org/#/c/167809/  merged.10:28
Kiallexcellent :) ekarlso and timsim have been doing similar perf/scale tests over the last little while10:29
KiallYou could just apply that patch to your deploy BTW, if you wanted to help test it before it lands :)10:29
stanzgyKiall: oh, I'm new to designate and not very familiar with the codes. I will check this later, thanks for the remind10:30
stanzgyokay10:31
KiallNo problem :) Generally, it looks like we don't use SQLA sessions/TX's because we try to keep all DB code in the storage module.. Since we're totally open to someone coming along with a NoSQL DB backend, and the rest of the app should in theory "just work" if they implement a storage driver that conforms to the API etc10:32
KiallMost of the other projects (for better or worse!) just decided SQLA was it, and that there would be no other choice..10:33
stanzgygot it, thx for the explanation. since I worked on nova and neutron before(which uses SQLA a lot), I'm a bit curious about the situation in Designate.10:34
KiallYea, I have to admit.. My decision 2-3 years ago to allow for non-SQLA stuff has bitten us more times than I can remember.. But I still think it was at least a mostly OK decision :)10:35
stanzgyyea, no problem about this now :D10:38
*** kei_yama has quit IRC10:45
*** stanzgy has quit IRC10:49
*** untriaged-bot has joined #openstack-dns11:02
untriaged-botUntriaged bugs so far:11:02
untriaged-bothttps://bugs.launchpad.net/designate/+bug/143724711:02
openstackLaunchpad bug 1437247 in Designate "FIP Reverse Zone is not owned by system when setting FIP" [Medium,New] - Assigned to Endre Karlson (endre-karlson)11:02
untriaged-bothttps://bugs.launchpad.net/designate/+bug/143706011:02
openstackLaunchpad bug 1437060 in Designate "FloatingIP PTR should have managed_tenant_id enforced" [Undecided,New]11:02
untriaged-bothttps://bugs.launchpad.net/designate/+bug/143707911:02
openstackLaunchpad bug 1437079 in Designate "pdns sync error" [Undecided,New]11:02
*** untriaged-bot has quit IRC11:02
*** GonZo2K has joined #openstack-dns11:17
*** GonZo2K has quit IRC11:24
*** km has quit IRC11:25
*** mwagner_lap has quit IRC11:38
*** johnbelamaric has joined #openstack-dns11:43
openstackgerritEndre Karlson proposed openstack/designate: Fix ownership issue with Zone's created on set fip  https://review.openstack.org/16831012:03
*** ericpeterson has joined #openstack-dns12:11
*** mwagner_lap has joined #openstack-dns12:22
*** ericpeterson has quit IRC12:31
*** jmcbride has joined #openstack-dns12:57
openstackgerritKiall Mac Innes proposed openstack/designate: Implement Pool Targets  https://review.openstack.org/16767813:51
*** ericpeterson has joined #openstack-dns14:06
openstackgerritKiall Mac Innes proposed openstack/designate: WIP: Add Healthcheck middleware  https://review.openstack.org/15135814:09
*** ericpeterson has quit IRC14:09
openstackgerritEndre Karlson proposed openstack/designate: Sync up FIP behaviour to match latest PoolManager  https://review.openstack.org/16835714:40
*** andrewbogott_afk is now known as andrewbogott15:04
*** ericpeterson has joined #openstack-dns15:04
*** ericpete_ has joined #openstack-dns15:06
*** ericpeterson has quit IRC15:06
*** MentalRay has joined #openstack-dns15:17
*** csoukup has joined #openstack-dns16:09
openstackgerritIhar Hrachyshka proposed openstack/designate: Fixed rootwrap file installation  https://review.openstack.org/16840216:17
*** crc32 has joined #openstack-dns16:22
*** MentalRay has quit IRC16:46
*** untriaged-bot has joined #openstack-dns17:02
untriaged-botUntriaged bugs so far:17:02
untriaged-bothttps://bugs.launchpad.net/designate/+bug/143707917:02
openstackLaunchpad bug 1437079 in Designate "pdns sync error" [Undecided,New]17:02
*** untriaged-bot has quit IRC17:02
*** pk__ has joined #openstack-dns17:41
*** jordanP has quit IRC17:45
*** mwagner_lap has quit IRC17:49
*** pk__ has quit IRC18:00
*** pk__ has joined #openstack-dns18:09
*** andrewbogott is now known as andrewbogott_afk18:10
*** andrewbogott_afk is now known as andrewbogott18:13
*** pk__ has quit IRC18:14
*** MentalRay has joined #openstack-dns18:20
*** penick has joined #openstack-dns18:31
*** penick_ has joined #openstack-dns18:35
*** penick has quit IRC18:36
*** penick_ is now known as penick18:36
*** crc32 has quit IRC18:46
*** pk__ has joined #openstack-dns18:50
*** andrewbogott is now known as andrewbogott_afk18:58
*** kfox1111 has joined #openstack-dns19:01
kfox1111ok. trying to setup automatic dns entry creation. doesn't look like I had nova setup right to send notifications. know off hand what daemons I need to restart?19:02
*** mwagner_lap has joined #openstack-dns19:25
kfox1111or something's consuming it. not really sure how this should work...19:29
kfox1111how does the notifications queue work?19:30
*** pk__ has quit IRC19:38
*** pk__ has joined #openstack-dns19:39
*** pk___ has joined #openstack-dns19:40
*** pk__ has quit IRC19:40
kfox1111is there an example config somewhere for sink?19:40
*** pk___ has quit IRC19:42
*** andrewbogott_afk is now known as andrewbogott19:45
andrewbogottkfox1111: I had to set notification_driver = nova.openstack.common.notifier.rpc_notifier in my nova conf before sink noticed anything19:47
andrewbogottkfox1111: my whole config should be public, lemme find some links19:48
kfox1111k. thanks for the help. :)19:49
andrewbogottafter you set that notifier (if it isn’t set for you already) you’ll want to restart nova-compute everywhere19:49
kfox1111yeah. I have notification_driver=nova.openstack.common.notifier.rpc_notifier and the ceilometer one too.19:50
kfox1111oh... its on all of the clients? ok.19:50
kfox1111that might explain why changes didn't seem to do anything. :)19:50
kfox1111well... on all the compute servers, I already have:19:52
kfox1111notify_on_state_change = vm_and_task_state19:52
kfox1111notification_driver = nova.openstack.common.notifier.rpc_notifier19:52
kfox1111but nothing's showing up in notifications.info. do I have to set anything else?19:52
andrewbogottkfox1111: ok, here’s my nova config:  https://github.com/wikimedia/operations-puppet/blob/production/modules/openstack/templates/icehouse/nova/nova.conf.erb19:53
andrewbogottand, here’s designate:  https://github.com/wikimedia/operations-puppet/blob/production/modules/openstack/templates/icehouse/designate/designate.conf.erb19:53
andrewbogottNote that I’m using a wacky home-made sink driver so you won’t be doing that :)19:54
andrewbogottkfox1111: I’m not sure I know what notifications.info is :)  I always just watch /var/log/designate/designate-sink.log19:55
kfox1111hmm... so other then the notification_driver=wikistatus thing, it seems about the same.19:55
kfox1111doing a rabbitmqctl list_queues | grep ^noti19:55
kfox1111shows two queues. notifications.error and notifications.info. 67 records in the first, 0 in the second.19:56
kfox1111I'm guessing designate would watch info.19:56
*** pk__ has joined #openstack-dns19:56
kfox1111Can designate sink watch one rabbit and talk to designate on another?19:56
andrewbogottkfox1111: I don’t know — I only use one rabbit19:57
andrewbogotthm, at least I thought I was :)19:58
* andrewbogott checks19:58
kfox1111I've got a rabbit for the cloud, but its juno and the rabbit is not exposed to the vm's. I am testing kilo (trunk) designate, so I setup its own rabbit in a vm to isolate everything.19:59
*** MentalRay_ has joined #openstack-dns19:59
kfox1111was first going to check that I had nova notifications working first, then figure out the best way to do the sink.19:59
andrewbogottDoes sink take a separate queue config from the designate general section?20:00
kfox1111not sure... I'll have a look.20:02
*** MentalRay has quit IRC20:02
kfox1111doesn't look like it. :/20:03
kfox1111though perhaps this might work: https://www.rabbitmq.com/shovel.html20:03
andrewbogottkfox1111: is the .info queue empty in your /production/ queue?  Or just in the mini vm one?20:09
kfox1111empty on the production one.20:11
andrewbogotthm, that seems unusual :)20:11
andrewbogottAlthough, I don’t know, is he queue emptied after a message is processed?20:12
kfox1111Normally, but that wouldn't make too much sense if you had a queue like notification where lots of parties might be interested in the same info?20:13
kfox1111normally it goes to the first client who can grab it when there are multiple workers.20:14
kfox1111hmm...20:14
kfox1111there are a lot of consumers listening... like: notifications.info      <rabbit@rcnu0.3.1204.0> 9       true    0       []20:15
kfox1111not sure how to relate consumers to clients though. :/20:15
*** pk__ has quit IRC20:15
*** pk__ has joined #openstack-dns20:22
kfox1111oh.. there can be multiple channels per client.20:26
*** pk__ has quit IRC20:26
*** pk__ has joined #openstack-dns20:33
kfox1111ah. it is ceilometer consuming messages from the queue.20:41
kfox1111I think.20:42
kfox1111its listening on the queue for sure.20:42
*** c_soukup has joined #openstack-dns20:42
kfox1111do you have both ceilometer and designate sink working together?20:42
andrewbogottI don’t use ceilometer.  so it’s possible that’s the issue20:43
andrewbogottalthough it would surprise me if that didn’t work :(20:43
*** csoukup has quit IRC20:45
kfox1111not sure. the way it looks like its setup out of the box, is for ceilometer to consume all messages. so if both were enabled, I think they would race, and half of the mssages would go to ceilometer and half to desigate. which would break both.20:45
kfox1111I think this is a case where you may need to add a fanout exchange?20:46
kfox1111or a topic exchange maybe...20:51
*** mfisch has joined #openstack-dns20:53
kfox1111ok. so notification.info does not show up in list_exchanges, so it must be a default type, which is direct.20:54
*** pk__ has quit IRC20:55
kfox1111so, yeah. it looks like its setup that way... :/20:55
*** mfisch has quit IRC20:56
*** mfisch has joined #openstack-dns20:56
*** mfisch is now known as Guest9095720:57
*** pk__ has joined #openstack-dns20:57
andrewbogottsounds like a good reason to file a designate bug20:59
-openstackstatus- NOTICE: Gerrit maintenance commences in 1 hour at 22:00 UTC http://lists.openstack.org/pipermail/openstack-dev/2015-March/059948.html21:00
*** pk__ has quit IRC21:02
kfox1111k.21:04
kfox1111not really sure if its a bug, or a lack of documentation, or what.. but wouldn't hurt to file a bug.21:05
kfox1111oh, bummer.... https://bugs.launchpad.net/designate/+bug/120470321:06
openstackLaunchpad bug 1204703 in Designate "tenant_name isn't accessible available for use in notification sink" [Wishlist,Confirmed]21:06
*** pk has joined #openstack-dns21:07
*** ericpete_ has quit IRC21:16
andrewbogottwait, it isn’t?  Because I’m using it21:18
andrewbogottoh, I guess I’m using tenant_id21:18
andrewbogottwhich in my case is the same21:19
kfox1111tenant_id isn't a uuid for you?21:19
kfox1111are you not using keystone?21:19
kfox1111ok. filed a bug. https://bugs.launchpad.net/designate/+bug/143749521:20
openstackLaunchpad bug 1437495 in Designate "sink notifications ceilometer" [Undecided,New]21:20
*** MentalRay_ has quit IRC21:20
andrewbogottkfox1111: I’m using keystone, and the ids are strings.  It’s probably a relic of my very very old setup21:24
*** MentalRay_ has joined #openstack-dns21:24
kfox1111really? hmm... just how old is your keystone? :)21:25
kfox1111its been a uuid for as long as I can remember... Since at least Essex?21:25
andrewbogottOur cloud is older than keystone, I think.21:25
andrewbogottCactus21:25
kfox1111ah. :)21:25
andrewbogottI guess I’ll keep an eye out for that… probably some future upgrade will break everything for me21:26
kfox1111you got me beet then. :)21:26
kfox1111or creating new tenants will not work the same as existing ones.21:26
* andrewbogott tries it21:27
andrewbogottnope, no uuids.  Running icehouse now21:27
andrewbogottmysterious21:27
kfox1111did you setup the keystone?21:27
kfox1111maybe its patched?21:28
andrewbogottI didn’t set it up but I’ve upgraded it several times.  It’s definitely running stock packages now.21:30
kfox1111odd. maybe a config option to make tenant_id=tenant_name?21:31
andrewbogottactually, I know what it is21:31
andrewbogottkeystone is backed with ldap, and is strictly a consumer21:31
andrewbogottI’m creating the projects, injecting them to ldap, and keystone has to take what it gets21:31
andrewbogottSo I bet if I create a project with Horizon, everything will break21:32
kfox1111oh. your using the ldap backend with tenants too?21:32
kfox1111that would do it.21:32
andrewbogottyep21:32
kfox1111did you hear they were talking about depricating that?21:33
andrewbogottI didn’t.21:33
andrewbogottThat would be…21:33
andrewbogottwell, not that big of a disaster, but a major drag.21:34
andrewbogottI guess I need to read more mailing lists :(21:34
kfox1111there's a thread on the devel mailing list from a month ago or so.21:34
kfox1111yeah. that part of the ldap driver never worked too well, and they are staring to add more and more featurs to tenants, so were talking about getting rid of that part. :/21:35
kfox1111With domains, they are starting to be able to give users control of domains, and they can create/manage their own tenants.21:35
andrewbogottah, so it’s just roles that they’re pulling out of ldap, not users21:35
kfox1111no, they are talking about supporting only users and groups in ldap.21:36
kfox1111tenants and roles would always be sql.21:36
andrewbogottHm, guess I’d better respond to that.21:37
kfox1111yeah.21:37
andrewbogottI wouldn’t mind moving tenants out of ldap, actually.  Although then I’ll run into that damn designate bug21:37
kfox1111yeah.21:41
kfox1111just need to add a keystone client into the code path to look up the name from the id. shouldn't be too hard a bug to fix.21:42
kfox1111fixing up your db to do tenant uuids would probably be harder.21:43
andrewbogottI create dns entries like <instance>.<tenant_name>.example.org21:43
andrewbogottSo I need it.21:43
kfox1111yeah. I was considering doing the exact same thing.21:44
andrewbogottEasier still would be for the notifier end to put tenant_name in the message payload.  That’d be a one-liner I bet.21:44
kfox1111for now though I keep runing up into sofware expecting to be amazon like and having hostname resolve between vm's in a tenant.21:44
kfox1111the nova sink would be good enough to fix that I think for now.21:45
kfox1111they don't do it today, since they don't have the info handy.21:45
kfox1111so either nova and neutron has to look it up, or designate does.21:45
andrewbogottdoesn’t dnsmasq handle at least that much?21:45
kfox1111only if your using nova-network.21:45
andrewbogottoh :(21:45
kfox1111yeah. :/21:46
andrewbogottWelp, another thing that neutron will break for me!21:46
andrewbogottThe future is looking better and better21:46
kfox1111hehe.21:46
kfox1111its been worth it. NaaS is awesome. :)21:46
kfox1111though it does take work to get working. :/21:46
andrewbogottIt still doesn’t support my use-case as far as I know21:46
andrewbogottNo floating ips21:46
kfox1111mostly that has been getting the network folks to figure out it works totally differently then what they are use to, and to just give me what I need and I'll get out of their hair. ;)21:47
kfox1111you don't want floating ip's? they are one of the 2 things in my bag of trics to make pets less pet like. they are really important I think.21:47
andrewbogottNo, I do want them.  I use flat dhcp w/floating IPs, last I checked neutron didn’t do that.21:48
kfox1111oh. yeah. I don't know if it does.21:48
kfox1111I always do tenant networks. then I can easily follow network traffic back to the tenant.21:49
kfox1111that made the network folks happy. I scared them to death saying users could setup their own machines. that was one of the only ways to calm them down.21:49
*** MentalRay_ has quit IRC22:03
-openstackstatus- NOTICE: Gerrit is offline for maintenance, ETA 22:30 UTC http://lists.openstack.org/pipermail/openstack-dev/2015-March/059948.html22:03
*** ChanServ changes topic to "Gerrit is offline for maintenance, ETA 22:30 UTC http://lists.openstack.org/pipermail/openstack-dev/2015-March/059948.html"22:03
*** pk__ has joined #openstack-dns22:16
*** pk has quit IRC22:16
*** MentalRay_ has joined #openstack-dns22:18
*** c_soukup has quit IRC22:19
*** ChanServ changes topic to "Zuul check queue stuck due to reboot maintenance window at one of our cloud providers - no need to recheck changes at the moment, they won't move forward."22:32
*** andrewbogott is now known as andrewbogott_afk22:39
*** pk__ has quit IRC22:44
*** MentalRay__ has joined #openstack-dns23:21
*** MentalRay_ has quit IRC23:24
*** MentalRay__ has quit IRC23:40

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!