uncategorized

DRY - Abstracting Gulp Tasks into Modules

Do you have gulp tasks you use in every project. I know I do. The source paths may be different but the tasks aren’t.
I was really tired of cutting and pasting those tasks in every project’s gulp file. Not to mention that if I wanted to change the task
a bit I had the nightmare of changing it everywhere. For example, we use babel to transpile code but need different options for client
code than we do for server code.

The pattern I am using now is:

  1. Always have task definitions in individual files (see an earlier post on unit testing).
  2. Create a project that has all the tasks and require it as a dependency.
  3. Make the tasks context agnostic and provide a configuration file to supply context.
  4. Expose the tasks in may main projects gulpfile so they can be used in the main applications flow.

Tasks as individual files

When you run a gulp task all you are really doing is invoking a function that task has defined. In most cases the return of that function is a stream.
Of course, gulp allows one task to depend on other tasks but you still run the actual by invoking a function.
Rather than using anonymous functions I use the following form:

1
gulp.task(ns+'copy', require("./gulp/defaultTasks/copy")(gulp, plugins, paths));

where ns+’copy’ is the task name (more on namescaping task names later.)
The calling program will provide context through “paths”
gulp and plugins are just references from the main gulp file:

1
var plugins = require('gulp-load-plugins')();
var gulp=require("gulp"
var paths = require('./setPathsGulp')(configFile);

What should your task module look like?

  1. It should return a function.
  2. When the function is invoked it should return a stream (or a thenable)
  3. For reusability it should not set context.

Here is an example:

1
'use strict';
module.exports = (gulp, plugins, paths)=>  ()=> {
	if (paths["debug"]) {
		return gulp.src(paths.src.copyIncludes, {base: paths.base}).pipe(plugins.debug()).pipe(gulp.dest(paths.dest));
	} else {
		return gulp.src(paths.src.copyIncludes, {base: paths.base}).pipe(gulp.dest(paths.dest));
	}
};

It is a very simple task. Copy source files to a destination.
The final “pipe” in gulp.dest returns a stream which will emit an end event when all the files have been copied.
Notice that it gets it’s source and destination from the ‘paths’ parameter which comes from a configuration file
created in the main app so it can be reused by an app that creates an appropriate configuration file.

##The structure of the task module.
We want to be able to access the tasks from our main applications. In order to do that we create an index.js file
which will be run when we “reguire” the module. This file will create actual gulp tasks from the individual files
and also create some additional tasks which are a combination of the individual tasks. In a moment we will see
how to expose these tasks in our applications main gulp file.

1
'use strict';
var u = require("./utilities/index");
module.exports =(gulp,ns,configFile)=> {
	if(ns && !u.endsWith(ns,".")){
		ns=ns+".";
	}
	ns=ns||"";
	var plugins = require('gulp-load-plugins')();
	var runSequence = require('run-sequence').use(gulp);
	var paths = require('./setPathsGulp')(configFile);
	gulp.task(ns+'clean', require('./gulp/defaultTasks/clean')(gulp, plugins, paths));
	gulp.task(ns+'copy', require("./gulp/defaultTasks/copy")(gulp, plugins, paths));
	gulp.task(ns+'babelFy', require('./gulp/defaultTasks/babelFy')(gulp, plugins, paths));
	gulp.task(ns+'babelFyIncludes', require('./gulp/defaultTasks/babelFyIncludes')(gulp, plugins, paths));
	gulp.task(ns+'copyIncludes', require('./gulp/defaultTasks/copyIncludes')(gulp, plugins, paths));
	gulp.task(ns+'watch', (cb)=> {
		runSequence(ns+'clean', ns+'copy', ns+'copyIncludes', ns+'babelFy', ns+'babelFyIncludes', [ns+'watchAll', ns+'watchBabelFy', ns+'watchBabelFyIncludes', ns+'watchCopyIncludes'], cb);
	});
	gulp.task(ns+'watchBabelFy', ()=> {
		var watcher = gulp.watch(paths.src.babelFy, {interval: 500}, (event)=> {
			if (paths.babeloptions.sourceMaps) {
				return gulp.src(event.path, {base: paths.base})
					.pipe(plugins.sourcemaps.init())
					.pipe(plugins.babel(paths.babeloptions))
					.pipe(plugins.sourcemaps.write('.'))
					.pipe(plugins.debug())
					.pipe(gulp.dest(paths.dest));
			} else {
				return gulp.src(event.path, {base: paths.base})
					.pipe(plugins.babel(paths.babeloptions))
					.pipe(plugins.debug())
					.pipe(gulp.dest(paths.dest));
			}
		});
	});
	gulp.task(ns+'watchBabelFyIncludes', ()=> {
		var watcher = gulp.watch(paths.src.babelFyIncludes, {interval: 500}, function (event) {
			if (paths.babeloptions.sourceMaps) {
				if (paths.debug) {
					return gulp.src(event.path, {base: paths.base})
						.pipe(plugins.sourcemaps.init())
						.pipe(plugins.babel(paths.babeloptions))
						.pipe(plugins.sourcemaps.write('.'))
						.pipe(plugins.debug())
						.pipe(gulp.dest(paths.dest));
				} else {
					return gulp.src(event.path, {base: paths.base})
						.pipe(plugins.sourcemaps.init())
						.pipe(plugins.babel(paths.babeloptions))
						.pipe(plugins.sourcemaps.write('.'))
						.pipe(gulp.dest(paths.dest));
				}
			} else {
				if (paths.debug) {
					return gulp.src(event.path, {base: paths.base})
						.pipe(plugins.babel(paths.babeloptions))
						.pipe(plugins.debug())
						.pipe(gulp.dest(paths.dest));
				} else {
					return gulp.src(event.path, {base: paths.base})
						.pipe(plugins.babel(paths.babeloptions))
						.pipe(gulp.dest(paths.dest));
				}
			}
		});
	});
	gulp.task(ns+'watchAll', ()=> {
		gulp.watch(paths.src.nowatch, {interval: 500}, function (event) {
			if (paths.debug) {
				gulp.src(event.path, {base: paths.base}).pipe(plugins.debug()).pipe(gulp.dest(paths.dest));
			} else {
				gulp.src(event.path, {base: paths.base}).pipe(gulp.dest(paths.dest));
			}
		});
	});
	gulp.task(ns+'watchCopyIncludes', ()=> {
		gulp.watch(paths.src.copyIncludes, {interval: 500}, function (event) {
			if (paths.debug) {
				gulp.src(event.path, {base: paths.base}).pipe(plugins.debug()).pipe(gulp.dest(paths.dest));
			} else {
				gulp.src(event.path, {base: paths.base}).pipe(gulp.dest(paths.dest));
			}
		});
	});
	gulp.task(ns+'default', [ns+'watch']);
	gulp.task(ns+'copyServer', require('./gulp/defaultTasks/copyOne')(gulp, plugins, paths,"server"));
	gulp.task(ns+'copyServerv4', require('./gulp/defaultTasks/copyOne')(gulp, plugins, paths,"server4"));
	gulp.task(ns+'copyClient', require('./gulp/defaultTasks/copyOne')(gulp, plugins, paths,"client"));

return paths}

Lets look at it

1
var paths = require('./setPathsGulp')(configFile);

This line calls a module which reads the configuration file and sets up our file paths. There is nothing
of import there. It checks to see if configFile refers to an actual file or is a json object which actually contains
the paths hash. If it is null (or the file reference doesn’t exist) it just returns the default hash.

1
gulp.task(ns+'clean', require('./gulp/defaultTasks/clean')(gulp, plugins, paths));
	gulp.task(ns+'copy', require("./gulp/defaultTasks/copy")(gulp, plugins, paths));
	gulp.task(ns+'babelFy', require('./gulp/defaultTasks/babelFy')(gulp, plugins, paths));
	gulp.task(ns+'babelFyIncludes', require('./gulp/defaultTasks/babelFyIncludes')(gulp, plugins, paths));
	gulp.task(ns+'copyIncludes', require('./gulp/defaultTasks/copyIncludes')(gulp, plugins, paths));

These lines create actual gulp tasks from the individual function definition files.
We also set up some watcher tasks to execute if changes in files are detected.

##Accessing the tasks from your applications gulp file
We only have one instane of gulp, the one that we instantiate in our main gulpfile. That is whay we
passed a reference to that to our module. The index.js, when it is creating tasks, is creating task on main
application’s instance of gulp.

1
'use strict';
var gulp = require("gulp");
require("gulp-babel-wrap")(gulp, "ems4", require("./serverConfigv4.json"));

The main gulp file is very simple. require gulp, require our module, and execute the function it exposes providing
a reference to gulp, the namespace, and the configuration.
Notice that in this case we just required a json file will return an actual configuration object. We could, of course,
have supplied a file name but since the module lives under node_modules we couldn’t use “./serverConfigv4.json” because
“.” in that case wouldn’t be the application directory. We could use path.resolve(“./serverConfigv4.json”) but it is simpler
to just retrieve the configuration object and pass it along.

These tasks are now available in your main application. Here is a screen shot from the Webstorm IDE
Gulp Tasks Webstorm

If you want to have different tasks for client code than for server code you can just call the module function
again with a different config file and namespace

'use strict';
var gulp = require("gulp");
require("gulp-babel-wrap")(gulp, "ems4", require("./serverConfigv4.json"));
require("gulp-babel-wrap")(gulp, "client", require("./clientConfig.json"));

##Summary
We can keep our code DRY by abstracting away common gulp tasks into their own module.
As long as we make the tasks context agnostic we can reuse them in multiple applications
supplying context in a config file. This keeps our application gulp file very simple yet
gives us the flexibility to add tasks for multiple contexts (server code running node and client code in the browser)
which may have, when transpiling, very different needs.

This is all part of the work I have been doing on a wrapper which allows you to transpile different parts of your code
with different options. For example you may want limited transpiling on the server but much more on the client code. Details
on that project can be found here: gulp-babel-wrap

Share