| *** goldyfruit has joined #openstack-dns | 02:16 | |
| *** goldyfruit has quit IRC | 02:24 | |
| *** nafiux has quit IRC | 06:49 | |
| *** nafiux has joined #openstack-dns | 07:06 | |
| *** pcaruana has joined #openstack-dns | 07:31 | |
| *** nafiux has quit IRC | 07:45 | |
| *** ircuser-1 has joined #openstack-dns | 07:57 | |
| *** ivve has joined #openstack-dns | 08:15 | |
| *** stingrayza_ is now known as stingrayza | 09:10 | |
| *** trident has quit IRC | 09:16 | |
| *** trident has joined #openstack-dns | 09:17 | |
| *** baffle has quit IRC | 09:22 | |
| *** altlogbot_0 has quit IRC | 09:24 | |
| *** altlogbot_0 has joined #openstack-dns | 09:24 | |
| *** baffle has joined #openstack-dns | 09:30 | |
| *** jawad_axd has joined #openstack-dns | 10:38 | |
| *** ivve has quit IRC | 12:34 | |
| *** jawad_axd has quit IRC | 13:08 | |
| *** ivve has joined #openstack-dns | 13:11 | |
| *** ivve has quit IRC | 13:16 | |
| *** ivve has joined #openstack-dns | 14:55 | |
| *** nafiux has joined #openstack-dns | 15:29 | |
| *** openstackgerrit has joined #openstack-dns | 15:35 | |
| *** ChanServ sets mode: +v openstackgerrit | 15:35 | |
| openstackgerrit | Erik Olof Gunnar Andersson proposed openstack/designate master: Removed deprecated pool-manager implementation https://review.opendev.org/665476 | 15:35 |
|---|---|---|
| *** ginopc has quit IRC | 16:01 | |
| *** goldyfruit has joined #openstack-dns | 16:13 | |
| *** ivve has quit IRC | 16:21 | |
| *** ivve has joined #openstack-dns | 16:52 | |
| *** ivve has quit IRC | 16:59 | |
| *** goldyfruit has quit IRC | 17:02 | |
| *** nafiux has quit IRC | 17:42 | |
| *** nafiux has joined #openstack-dns | 17:45 | |
| *** trident has quit IRC | 17:47 | |
| *** trident has joined #openstack-dns | 17:51 | |
| *** goldyfruit has joined #openstack-dns | 17:59 | |
| *** JonathanD has quit IRC | 18:19 | |
| *** JonathanD has joined #openstack-dns | 18:37 | |
| *** nafiux has quit IRC | 19:17 | |
| *** abaindur has joined #openstack-dns | 19:28 | |
| *** nafiux has joined #openstack-dns | 19:30 | |
| *** goldyfruit has quit IRC | 19:37 | |
| *** goldyfruit has joined #openstack-dns | 20:16 | |
| openstackgerrit | Corey Bryant proposed openstack/designate master: Ensure coordination IDs are encoded https://review.opendev.org/673360 | 20:17 |
| *** trident has quit IRC | 20:49 | |
| *** trident has joined #openstack-dns | 20:52 | |
| openstackgerrit | Erik Olof Gunnar Andersson proposed openstack/designate master: Removed deprecated pool-manager implementation https://review.opendev.org/665476 | 21:05 |
| *** goldyfruit has quit IRC | 21:07 | |
| abaindur | trying to understand DNS serials a bit here, can anyone help me understand why our zone is repeatedly going into ERROR state? | 21:20 |
| abaindur | so our rndc is failing to addzone due to Stderr: u"rndc: 'addzone' failed: already exists\n" since this was a zone that already existed | 21:21 |
| abaindur | afterwards we see: Letting worker send NOTIFYs instead notify_zone_changed and Sent NOTIFY to 10.195.0.121:53 for zone grafana.infra. | 21:22 |
| eandersson | Maybe you have a duplicate entry or similar in the pools.yaml config? | 21:22 |
| abaindur | but it fails on polling. we have two nameservers listed in our pool.syaml | 21:22 |
| abaindur | they are returning different sderials? | 21:22 |
| abaindur | Found serial 1563914415 on 10.195.0.121 for zone grafana.infra. __call__ | 21:22 |
| abaindur | Found serial 0 on 10.195.0.122 for zone grafana.infra. __call__ | 21:22 |
| abaindur | Results for polling grafana.infra.-1563914415: DNSQueryResult(positives=1, no_zones=1, consensus_serial=1563914415, results=[1563914415, 0]) | 21:22 |
| abaindur | Unsuccessful poll for grafana.infra. on attempt 4 _do_poll | 21:23 |
| eandersson | Maybe the domain already existed on bind | 21:23 |
| abaindur | basically it then repeatedly gets stuck in a loop like that - polling 10 times, then trying the rndc, then polling 10 times, etc.. | 21:23 |
| eandersson | Would it be possible to delete the zone in Designate, and make sure it is removed in bind? | 21:23 |
| eandersson | and then re-add it | 21:23 |
| abaindur | Yes, this zone already existed. the hosts where designate was installed on got shuffled around | 21:23 |
| abaindur | the zone was already created by Designate and has many records | 21:24 |
| eandersson | I see | 21:24 |
| abaindur | don't want to delete it, is there another way to recover? | 21:24 |
| abaindur | am i correct in guessing that its failing because one NS returns a serial, the other returns a serial of 0? | 21:25 |
| eandersson | mugsie, ^ | 21:26 |
| eandersson | Maybe the zone is misconfigured on 10.195.0.122 (e.g. got the wrong master, should be your designate ip) | 21:26 |
| eandersson | What happens if you do | 21:28 |
| eandersson | dig @10.195.0.122 grafana.infra AXFR | 21:28 |
| eandersson | does it list the records? | 21:28 |
| *** nafiux has quit IRC | 21:43 | |
| *** nafiux has joined #openstack-dns | 22:02 | |
| abaindur | eandersson: sorry was in meeting | 22:20 |
| abaindur | it is saying Transfer failewd | 22:20 |
| abaindur | ; Transfer failed | 22:20 |
| eandersson | And if you do it against 121? | 22:20 |
| abaindur | Same | 22:20 |
| eandersson | I'll try to set up bind9 tonight at home and see if I can reproduce it | 22:21 |
| eandersson | mugsie, is probably sleeping already, but I am sure he would know as well | 22:21 |
| abaindur | yea np, I don't have the most expertise with bind myself either, besides a basic setup | 22:22 |
| abaindur | It is only happening with this one zone. there are other zones that show ACTIVE | 22:22 |
| abaindur | eandersson: based on those DNSquery logs above, | 22:28 |
| abaindur | Results for polling grafana.infra.-1563914415: DNSQueryResult(positives=1, no_zones=1, consensus_serial=1563914415, results=[1563914415, 0]) | 22:28 |
| abaindur | does that mean designate is polling two backend nameservers to see if the change was reflected | 22:28 |
| abaindur | consensus serial is the serial# designate is looking for? | 22:28 |
| eandersson | Yea - I think it's saying that one of them isn't set up properly, or maybe that the server is denying the request | 22:29 |
| abaindur | and results=[1563914415, 0] indicated the response from each? | 22:29 |
| eandersson | In the bind config you should have masters configured for that zone | 22:29 |
| eandersson | and maybe the ips are wrong | 22:29 |
| abaindur | hmm, let me see if we can remove the secondary nameserver from pools.yaml? | 22:29 |
| abaindur | Found serial 1563914415 on 10.195.0.121 | 22:29 |
| abaindur | Found serial 0 on 10.195.0.122 | 22:29 |
| eandersson | > { type slave; masters { ;}; file "slave.grafana.infra.6f2a845e-4e27-4f5c-a40d-171e2f40e3e5"; }; | 22:29 |
| eandersson | You should have a file like this | 22:29 |
| eandersson | on the bind9 side | 22:30 |
| eandersson | and thinking that maybe the masters portion isn't set properly | 22:30 |
| abaindur | pools.yaml is listing both those hosts under the nameservers: section | 22:30 |
| eandersson | how about masters | 22:30 |
| eandersson | Are all your designate servers listed there? | 22:31 |
| abaindur | Is that in the named.conf file? | 22:31 |
| eandersson | sorry under pools.yaml | 22:31 |
| eandersson | https://github.com/openstack/designate/blob/master/devstack/designate_plugins/backend-bind9#L103 | 22:31 |
| abaindur | ah no... both IPs are listed under the nameservers: section | 22:34 |
| abaindur | but under targets: | 22:34 |
| abaindur | our masters: section lists the IPs of our two hosts where Designate is running | 22:35 |
| abaindur | and the options: section only has the host/rndc_host: for the .121 IP | 22:35 |
| abaindur | (.121 and .122 are located elsewhere, on a different backend machines) | 22:36 |
| abaindur | is the options: under targets: supposed to list all rndc targets for all nameservers? | 22:37 |
| abaindur | actually the .122 is listed under the also_notifies: section | 22:40 |
| eandersson | Under options I think you can/should only target one, but it might be worth trying adding a second target | 22:44 |
| eandersson | just to see if that resolves it | 22:46 |
| *** abaindur has quit IRC | 23:27 | |
| *** abaindur has joined #openstack-dns | 23:30 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!