How to implement gdnsd on FreeBSD
I was looking for a solution to increase the availability of my public web sites, see my previous post for the introduction to this.
gdnsd is an authoritative-only name server. The initial ‘g’ stands for geographic, as gdnsd offers a plugin system for geographic (or other sorts of) balancing, redirection, and service-state-concious failover. This post explains how gdnsd is implemented at SoCruel.NU to achieve the availability goals, which are:
- have a failover possibility in place for the public SoCruel.NU web sites in case the web sites become unavailable on the primary location in cases of i.e. internet connection down or server hardware failure
- a failover must take place in a couple of minutes (2 to 3 minutes - so this amount of down time is accepted)
To make these goals happen the simplefo gdnsd plugin is used. This is one of the more simple plugins of gdnsd. More complex setups with geographic balacing, redirection and service-state-concious failover scenario’s are possible!
Requirements
The following requirements have to be in place to be able to implement what is described in this post:
- have a public DNS zone running on the internet with full administrative control - for this blog post this is assumed to be
mydomain.tld - the name servers of the public DNS zone
mydomain.tldare:ns1.mydomain.tld,ns2.mydomain.tldandns3.mydomain.tld - have 2 up to date FreeBSD systems running with a direct connection on the internet with a public IP address (assumed 2.4.6.8 and 3.5.7.9, respectively, for this post) for running gdnsd
- the FQDN of these 2 gdnsd servers are:
gdns1.mydomain.tldandgdns2.mydomain.tld - have 2 web servers running with, assumed for this blog post, public IP addresses
11.12.13.1and22.33.44.5. These web servers run the public web sitewebapp.mydomain.tld.
Please replace all assumed names and IP addresses with your own, if you want to implement this in your own environment.
Install gdnsd (version 2)
gdnsd has currently 2 major versions: version 2.x and version 3.x. Both are available in the FreeBSD ports and packages collection.
Use the following command to install the gdnsd package on the gdns1.mydomain.tld and gdns2.mydomain.tld systems:
$ sudo pkg install gdnsd2
The installation package has created the directory /usr/local/etc/gdnsd, which is where gdnsd stores its configuration. We create two new directories here to store our plugin and service type configurations:
$ sudo mkdir /usr/local/gdnsd/plugins.d $ sudo mkdir /usr/local/gdnsd/service-types.d
Before we start the configuration of gdnsd we create the admin_state file (otherwise gdnsd complains at startup):
$ sudo touch /var/db/gdnsd/admin_state
For more information about the admin_state see the gdnsd wiki.
Configure gdnsd (version 2)
The configuration of this gdnsd implementation consist of 4 parts:
- the gdnsd main configuration file
- the gdnsd service type configuration
- the gdnsd plugin configuration
- the gdnsd zone file
Each of these is discussed below. All 4 must be performed on both the gdns1.mydomain.tld and gdns2.mydomain.tld systems. The configuration of gdns1.mydomain.tld is shown.
gdnsd main config file
The main gdnsd configuration file on FreeBSD is /usr/local/etc/gdnsd/config. This configuration file is clearly documented on the gdnsd GitHub wiki. The main configuration file used for this implementaion looks like:
options => {
listen => [ 2.4.6.8, 127.0.0.1 ]
dns_port => 53
http_listen => [ 127.0.0.1 ]
http_port => 3506
chaos_response => "Just a nameserver."
}
service_types => {
$include{service-types.d}
}
plugins => {
simplefo => {
$include{plugins.d}
}
}
The following is configured here:
- the gdnsd daemon listens on the IP addresses 2.4.6.8 and localhost on port 53
- the gdnsd web interface listens on localhost port 3506
- the DNS chaos record is set to “Just a nameserver”
- the gdnsd service type configuration files in the directory
/usr/local/etc/gdnsd/service-types.dare loaded - the simplefo plugin is used
- the gdnsd plugin configuration files in the directory
/usr/local/etc/gdnsd/plugins.dare loaded
gdnsd service type configuration
With a gdnsd service type you define how your end points, in this case the web servers, are monitored by gdnsd. The status is shown on the gdnsd web interface (see the main config file).
The name of the service type for our webapp.mydomain.tld URL is http_monitor_webapp and its configuration is:
http_monitor_webapp => {
plugin => http_status
ok_codes => [200]
vhost => webapp.mydomain.tld
url_path => /monitor.html
interval = 15
timeout = 5
up_thresh = 5
ok_thresh = 3
down_thresh = 6
}
This is copied and pasted in a file called webapp.conf and put it in the directory /usr/local/etc/gdnsd/service-types.d. This configuration states:
- that we check our web servers using the HTTP protocol
- only the HTTP code 200 is valid for our checks
- we check for the virtual host URL
webapp.mydomain.tld - we check for the file
monitor.html - we use an interval (“number of seconds between successive monitoring requests for a given resource”) of 15 seconds between checks
- the timeout (“Maximum time the monitoring code will wait for a successful response before giving up and considering the request to be a failure.”) is 5 seconds
- the up threshold (“Number of monitoring requests which must succeed in a row without any failures to transition a given resource from the DOWN state to the UP state.”) is 5
- the down threshold (“Failure counter to change to DOWN state. If the failure counter reaches down_thresh the resource is transitioned to the DOWN state.”) is 6
- the ok threshold (“if ok_thresh successes occur in a row with no failures between them, the failure counter is reset back to zero.”) is set to 3
For more detailed information about the gdnsd service types see the gdnsd GitHub wiki.
gdnsd plugin configurations
With a gdnsd plugin you define the failover type of the service you load balance. gdnsd ships with 8 plugins. Here the simple primary -> secondary address failover is chosen using the simplefo plugin. Its configuration is:
webapp => {
service_types => http_monitor_webapp
primary => 11.12.13.1
secondary => 22.33.44.5
}
This is copied and pasted in a file called webapp.conf and put it in the directory /usr/local/etc/gdnsd/plugins.d. This configuration states:
- that the service type
httpmonitorwebappis used - the primary web server is the server with IP address
11.12.13.1 - the secondary (failover) web server is the server with IP address
22.33.44.5
the zone file
The gdnsd zone files reside in the directory /usr/local/etc/gdnsd/zones. A file named webapp.mydomain.tld is created in this directory and the contents are as below:
$TTL 3600
$ORIGIN webapp.mydomain.tld.
@ IN SOA gdns1.mydomain.tld. hostmaster.mydomain.tld. (
2020042201 ; serial number
43200 ; Refresh
3600 ; Retry
864000 ; Expire
7200 ; Min TTL
)
60 IN NS gdns1.mydomain.tld.
60 IN NS gdns2.mydomain.tld.
@ 15 DYNA simplefo!webapp
A gdnsd zone file looks very much like a zone file of any other name server software. Except for the last line:
@ 15 DYNA simplefo!webapp
This is a DYNA record, which is for dynamically-determined address records (both A and AAAA) via plugin code. This line states the following:
- this is a blank record (the ‘@’ character), which points the domain name, here
webapp.mydomain.tld, to a server (here one of the web servers) - the
TTLvalue is 15 seconds - the
simplefoplugin is used for this record - the resource name of the plugin (here
webapp)
Configure the mydomain.tld zone
Once you have done the above, one more item needs to be configured. And that is configure the name servers for the webapp.mydomain.tld domain in the mydomain.tld. zone (which runs on the ns1.mydomain.tld., ns2.mydomain.tld. and ns3.mydomain.tld. name servers). This is done in the last 2 lines of the example zone file shown below:
$ORIGIN mydomain.tld. ; default zone domain
$TTL 3600 ; default time to live
@ IN SOA ns1.mydomain.tld. hostmaster.mydomain.tld. (
2020042201 ; serial number
43200 ; Refresh
3600 ; Retry
864000 ; Expire
86400 ; Min TTL
)
IN NS ns1.mydomain.tld.
IN NS ns2.mydomain.tld.
IN NS ns3.mydomain.tld.
webapp IN NS gdns1.mydomain.tld.
IN NS gdns2.mydomain.tld.
This example zone file can be used for name servers like BIND or NSD.
A failover explained
So what happens when a failover occurs for webapp.mydomain.tld from the primary web server to the secondary web server in case the primary web server becomes unavailable?
The name servers for the webapp.mydomain.tld zone have a time to live value configured of 60 seconds. gdnsd ‘knows’ that the primary web server is down after 6 x 15 seconds is 90 seconds. So after the next (the 7th) interval gdnsd answers queries with the IP address of the secondary (failover) web server.
But the clients have to wait a little longer because of the time to live value of 60 seconds, as they will get an update after 120 seconds.
As soon as the primary web server becomes available again gdnsd takes up_thresh times interval seconds to change the query answers back using the primary web server IP address. The clients still depend on the time to live value of 60 seconds. But that is less important in this case, as now no downtime is part of this!
Resources
Some (other) resources about this subject: