uncategorized

Updating CloudFlare DNS Records Dynamically Using Node, Consul and Nginx

In an earlier post nginx I discussed updating Nginx configuration dynamically when the domains it controls changes. Today I would like to discuss leveraging that configuration to update DNS records. The implementation discussed uses the CloudFlare api but the approach could be used for any DNS provider which has an api.

Joyent has an example for doing this here.

In their example they have a CloudFlare container and use CloudFlares restful api via a bash script to handle the updates.

My implementation is a bit different:

  1. I have no CloudFlare container.
  2. I add node to the nginx container.
  3. I use node to handle the updates using the cloudflare4 package.

Detecting Nginx changes

ContainerBuddy and Consul work together to detect when changes occur and when necessary regenerate the nginx.conf file by running the following bash script on the nginx container

1
#!/bin/bash

if [ -z "$VIRTUALHOST" ]; then
    # fetch latest virtualhost template from Consul k/v
    curl -s --fail consul:8500/v1/kv/nginx/template?raw > /tmp/virtualhost.ctmpl
else
    # dump the $VIRTUALHOST environment variable as a file
    echo $VIRTUALHOST > /tmp/virtualhost.ctmpl
fi

# render virtualhost template using values from Consul and reload Nginx
consul-template \
    -once \
    -consul consul:8500 \
    -template "/tmp/virtualhost.ctmpl:/etc/nginx/conf.d/default.conf:nginx -s reload"

/usr/local/bin/node  /application/index.js --task updatedns

The consul template which generates the conf file is stored as a key/value pair in consul. The script just fetchs that template and stores it in the tmp directory, renders the template and streams it to the appropriate place in the nginx file system. It then reloads nginx.

The last line of the script is where we call a node script to update the dns.

Updating DNS via Node

The node application:

  1. Retrieves the ip address of nginx.
  2. Retrieves the domains it controls from a Manta file.
  3. Filters the domains to those for currently active services.
  4. Filters the domains to those managed by CloudFlare.
  5. If the ip on CloudFlare is the same as Nginx continue to next domain otherwise:
  6. Call the CloudFlare api to update (or add) the nginx ip to the appropriate A record for the domain.

Code snippets:

1
let updateDNS = ()=> {
	isFilePresent()
		.then((configObj)=> {
			getActiveServices()
				.then(()=> {
					processCloudFlare(configObj);
				})
				.catch((err)=> {
					process.exit(1);
				})
		})
		.catch((err)=> {
			//console.log(err, err.stack);
			retrieveFromManta()
				.then(()=> {
					isFilePresent()
						.then((configObj)=> {
							getActiveServices()
								.then(()=> {
									processCloudFlare(configObj);
								})
								.catch((err)=> {
									process.exit(1);
								})
						})
						.catch((err)=> {
							//console.log(err);
							process.exit(1);
						})
				})
				.catch((err)=> {
					//console.log(err);
					process.exit(1);
				})
		})

}

isFilePresent checks to see if we have a local copy of the configuration. If not we retrieve the file from the Manta store:

1
let retrieveFromManta = ()=>new Promise(function (resolve, reject) {
	let client = manta.createClient({
		sign: manta.privateKeySigner({
			key  : fs.readFileSync(__dirname + '/id_rsa', 'utf8'),
			keyId: process.env.MANTA_KEY_ID,
			user : process.env.MANTA_USER
		}),
		user: process.env.MANTA_USER,
		url : process.env.MANTA_URL
	});
	client.get('~~/stor/dns/dnsConfig.json', (err, stream)=> {
		if (err) {
			//console.log(err);
			reject(err);
		} else {
			stream.pipe(fs.createWriteStream("/tmp/dnsConfig.json"));
			stream.setEncoding('utf8');
			stream.on('end', resolve);
			stream.on('error', reject);
		}
	});
});

This piece just authenticates to Manta, retrieves the config file and writes it to the local file system.

Once we have the config file we are ready to examine the CloudFlare records:

1
let processCloudFlare = (config)=> {
	config = JSON.parse(config);
	let api = new CloudFlareAPI({
		email                    : process.env["CF_AUTH_EMAIL"],
		key                      : process.env["CF_API_KEY"],
		itemsPerPage             : 100,
		maxRetries               : 5,
		raw                      : false,
		autoPagination           : false,
		autoPaginationConcurrency: 1
	});
	api.zoneGetAll()
		.then((zones)=> {
			let zoneInfo = zones.map((zone)=> {
				return {
					id  : zone.id,
					name: zone.name
				}
			});
				aAliases = [];
				Object.keys(config).forEach((key)=> {
					if(activeServices && activeServices.indexOf(key)!== -1) {
						config[key].serverAlias.forEach((sa)=> {
							aAliases.push(sa);
						});
					}
				})
			let promises = [];
			//now we loop over all the aliases and if they are in a cloudflare zone we either add or update the record
			aAliases.forEach((alias)=> {
				let aParts = alias.split(".")
				let zn = null;
				let zid = null;
				let pl = aParts.length;
				if (pl >= 2) { // get zone name
					zn = aParts[pl - 2] + "." + aParts[pl - 1];
				}
				if (zn) {
					let isManaged = false;
					zoneInfo.forEach((zon)=> {
						if (zon.name === zn) {
							isManaged = true;
							zid = zon.id;
						}
					});
					if (isManaged) {
						promises.push(updateDNSRecord(api, zn, zid, alias))
					}
				}
			});
			Promise.all(promises)
				.then(()=> {
					process.exit(0)
				});
		})
		.catch((err)=> {
			console.log(err, err.stack);
			process.exit(1);
		})
}

We authenticate to CloudFlare and get a list of all the domains it manages. We then loop over all the serverAlias’s in the config file, making sure the domain is managed by CloudFlare.

We then update the dns records. Since updating a record is asynchronous we create an array of promises and then resolve this function’s promise when they have all completed using Promise.all (We use the bluebird library for promises).

Updating the dns record is pretty simple. We just fetch the current record from CloudFlare and if not found add it. If it is found and has an ip different than the nginx ip we update it.

1
let updateDNSRecord = (api, zonename, zoneid, recordName)=> new Promise(function (resolve, reject) {

	let doIt = ()=> {
		//see if record is there
		api.zoneDNSRecordGetAll(zoneid, {name: recordName, type: "A"})
			.then((records)=> {
				if (records.length) { //we found it
					//need.to update record
					//note this will not work if we have multiple ip's in dns TODO: handle
					let record = records[0];
					if(record.content===nginxIP){
						resolve(true); //already current
					}else {
						record.content = nginxIP;
						delete record.meta;
						delete record.created_on;
						delete record.modified_on;
						delete record.zone_name;
						delete record.zone_id;
						delete record.locked;
						delete record.proxied;
						delete record.proxiable;
						let id = record.id;
						delete record.id;
						api.zoneDNSRecordUpdate(zoneid, id, record, true)
							.then((result)=> {
								//console.log(result, record.name);
								resolve(true);
							})
							.catch((err)=> {
								//console.log(err, err.stack);
								reject(err);
							});
					}
				} else { //need a new record
					api.zoneDNSRecordNew(zoneid, {
							name   : recordName,
							type   : "A",
							content: nginxIP,
							ttl    : 1
						})
						.then((results)=> {
							resolve(true)
						})
						.catch(reject);
				}
				//we need to create one

			})
			.catch(reject);
	}
	if (!nginxIP) {
		getNginxIP()
			.then((results)=> {
				nginxIP = results;
				doIt();
			})
			.catch(reject);
	} else {
		doIt();
	}
});

The entry point is at the bottom of the function where we make sure we have the nginx ip before calling the internal function doit.
Notice that we have to delete some attributes from the received CloudFlare record because we cannot change them and the api rejects the request if they are included.

1
let getNginxIP = ()=> new Promise(function (resolve, reject) {
	let client = smartdc.createClient({
		sign: smartdc.privateKeySigner({
			key  : fs.readFileSync(__dirname + '/id_rsa', 'utf8'),
			keyId: process.env.SDC_KEY_ID,
			user : process.env.SDC_ACCOUNT
		}),
		user: process.env.SDC_ACCOUNT,
		url : process.env.SDC_URL
	});
	client.listMachines((err, machines)=> {
		if (err) {
			//console.log(err, err.stack);
			reject(err);
		} else {
			let results = machines.reduce((prev, curr)=> {
				if (curr.name.indexOf("nginx") !== -1) {
					prev = curr.primaryIp;
				}
				return prev;
			}, null);
			//console.log(`The nginx ip is ${results}`);
			resolve(results);
		}
	})
})

We use Joyent’s sdc module to get the nginx ip. We just get a list of currently running machines and find nginx. Note that we cannot use “docker ps” for this because this code is running in a docker container and has no docker daemon running.

In the previous post we discussed configuring nginx and in this post leveraged that to update dns. In the next post we will tie it all together.

Share