Git: Node Pre-commit Hooks

Background

Git hooks allow scripts to be fired off when important events such as committing a change to the repository occur. This guide details how to set up a pre-commit hook for a node application that will execute a testing task that must pass before any new commits are added into the repository.

Why?

Pre-commit hooks allow house-keeping tasks such as code linting and running tests to execute before the commit is added to the repository. If any of these tasks fail then the commit is rejected and not added into the repository. Doing this helps enforce code integrity and prevents simple easy to catch mistakes from creeping in. Of course, any other other kind of tasks could be run like emailing your boss that you’re about to commit, and there are other hooks that can be used for post-commits.

Set up

For this guide, I’ve set up a very simple node command line calculator application that can be cloned and used to follow along with the rest of the article.

The app takes exactly two whole number parameters and sums them together to output the result in the console.

This app has a small test suite that can be run by executing the following command.

Currently, all the tests are passing.

New feature request

Scenario: Your boss comes over to you and says “that calculator app is great… but I urgently need to be able to subtract numbers as well as add. Make it accept an operator as a parameter”.

So the new feature is loosely defined, we could go rushing ahead make changes to the code and commit it immediately. But what about the tests? Well they would be broken because in our haste we forgot to run the tests before committing. Now other team members are upset with us and the build server is sending you emails that you have broken everything. Not good.

If only the tests had been run automatically before the commit was made. And that’s where pre-commit hooks come in.

Adding pre-commit hooks

First set up the git pre-commit hook that will enforce running our tests and making sure they pass before committing to the repository.

The node package “pre-commit” allows the defining of tasks in package.json that will be run before making the commit. If any of these tasks return an error code then the commit will be aborted.

To install:-

Installing pre-commit will add a new task called “pre-commit” into the hidden .git/hooks folder located under the root of the project. It is this script that git will execute before committing to the repository. If the script returns a non-zero value, then the commit will be aborted.

To configure a test task to execute on pre-commit, add the following block into package.json:-

silent reduces the amount of noise that pre-commit logs out to the terminal.

run takes an array of npm tasks to execute. These tasks are defined within the “scripts” block in the package.json file.

By default, npm test will be called if no run tasks are defined, but chances are that more tasks such as linting will be added the project so I prefer explicitly defining this upfront. Read the pre-commit documentation for more information.

That’s it. Now whenever anybody commits changes to the repository, the tests will execute. If the tests return a non-zero result, the commit will be abandoned with an error message.

Adding the feature

To add the subtract feature to the code, I’ve created a gist that you can copy the changes from. Replacing the contents of calc.js with this gist will add the subtract function into the program.

Now the user needs to enter 3 arguments rather than two to operate this simple app.

Giff diff report showing the changes made to calc.js to add the subtract feature
Giff diff report showing the changes made to calc.js to add the subtract feature

In the terminal, the user can now execute the following command to add or subtract two numbers:-

Terminal output showing program running addition and subtraction operations.
Terminal output shows that the calc app now can add and subtract numbers together

Great! The changes work, feature complete. Time to commit.

Git prehook commit failure message
Terminal output showing the broken test caused by adding the new feature in.

The commit was rejected because the tests are no longer passing. Excellent, the tests were automatically run and have stopped the feature from accidentally being committed to the code base.

Now all that remains to be done is to fix the existing tests and add a new test for the new subtract feature, then commit. Check out the subtract-feature branch on the repo for the solution.

Git pre hook success message
Terminal output showing that are tests are now passing before making the commit

Further reading

You can find more information about git hooks on https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks

Debugging Performance Bottlenecks in a Node.js Application

Background

A recent update to one of our high volume production apps had the undesired consequence of increasing the time-to-first-byte by a staggering 500%. Although we had done rigorous testing, this bug did not clearly manifest itself until it was in production. To make matters worse, the application is isomorphic and we had just updated from React v0.13 to v0.14 refactoring components along the way so there was not one single change we could easily pinpoint as being the most probable cause. Tasked with identifying the problematic code, I began CPU profiling the JavaScript node application using the readily available tool – node-inspector.

This guide aims to demonstrate how to diagnose bottlenecks in your node application using node-inspector and Google Chrome.

Sample project

I’ve put together a very basic express app that calls off to a fake service which adds a delay of ~1 second (depending on your machine) to the response from the server. The code can be cloned from github:-

Quick start

To start the server, execute the following from the application root:-

The server should then output the following response:-

Node performance debug site server listening console output
Server ready and listening

Identifying the problem

Next, open up Chrome and with debugging tools enabled and focused on the network tab, navigate to http://localhost:3000/

Network trace for the node performance debug site showing a 1 second delay for time-to-first-byte
Network trace of our sample node application showing a delay of over a 1 second for time-to-first-byte.

We can clearly see that something is not right; over 1 second to get a response is too slow. So how can we diagnose what’s causing the delay?

CPU profilling Node.js

Node inspector Can help us to identify bottlenecks in the execution of the node application by profiling the code execution.

First install node-inspector globally (you may need to run as sudo),

then execute the node application using node-debug instead of node.

This will launch your application in node-inspector and open a Chrome debugger window.

Application console.log running under node-inspector
Application console.log running under node-inspector

If your browser does not automatically load, then open up a new browser instance and navigate to the debug url displayed in the console e.g. http://127.0.0.1:8080/?ws=127.0.0.1:8080&port=5858

node-inspector Chrome debug window
node-inspector Chrome debugger attached to the process and paused at the first line of code

By default, node inspector will pause execution of the application at the very first line of code.

Allow code execution to continue by pressing F8 or clicking the resume execution button.

Recording the CPU Profile

To record the JavaScript CPU profile, we need to instruct the node inspector debugging tools to record the CPU profiles, then in another browser window visit our site to generate a request that will appear in the CPU profile.

Step 1. Profiles tab

Start CPU profiling in node inspector
Start CPU profiling in node inspector

Step 2. Make sure the “Collect JavaScript CPU Profile” option has been selected.

Step 3. Click start.

Step 4. In a separate browser, navigate to http://localhost:3000/ and wait for the request to complete.

Step 5. Back in node inspector, click “Stop”.

Stop recording of the CPU profile in node inspector
Stop CPU profile recording in node inspector

6. The profile will automatically load.

Analysing the CPU profile graph

The CPU profile chart helps to easily identify where our application is spending its time whilst processing the request.

1. Make sure the chart view is selected.

Annotated node inspector CPU profile trace
The node inspector CPU profile trace – Note that the duration of the call time has increased from 1 second to almost 1 minute. Running with a debugger attached slows down the application significantly.

2. The graph shows the activity over time. We’re interested in this big block.

3. The call stack starting from top to bottom.

Looking at the call stack, we can quickly identify where our application is spending most of its time processing the request.

Using the mouse to hover over each method call, we can see that it looks like the culprit lies in slowService.js, around line 14. Armed with the knowledge let’s go to the code and see what’s going on.

So the method that’s taking a lot of time to execute is appropriately named cpuIntensiveProcess.

Clicking on the block with the name “service” will directly take you through to the source code whilst inside node inspector.

Node inspector source view of slowService.js
Viewing the problematic file in node inspector

In our sample application we can now refactor the cpuIntensiveProcess method out to be more performant, or in this example just delete it entirely because it’s not doing anything useful.

Conclusion

It is relatively pain free to troubleshoot performance problems in our application. Node-inspector is just one of a suite of tools available to help us monitor, report and diagnose performance problems.

In our production environment, we use newrelic to alert if there is ever a significant degradation in time-to-first-byte performance, as well as capturing server and client side errors giving better insights into how our applications behave in the real world.

Fixing node package vulnerabilities

It’s always a good idea to keep on top of your security updates because if you’re not looking at it, someone else probably will.

Acting on a recent denial of service vulnerability security alert from Node.js, I was tasked with upgrading our production servers to the latest Node.js version to fix the security alert. This was pretty straight forward upgrade only involving changing a version number in a config file then rerunning the build / production pipelines in teamcity to pull in the latest Node.js.

Following on from this I wanted to check if our applications themselves contained any security vulnerabilities, the only problem is that in just one project we have at least 30 top level package dependencies defined in our package.json and each referenced package references other packages and so on. Reviewing these packages by hand would take hours and is highly error prone to missing known vulnerabilities.

Thankfully there’s a tool to do all the hard work for you by scanning your node package.json and npm-shrinkwrap.json files for vulnerabilities.

nsp from nodesecurity.io audits your node application dependencies against its database of known security vulnerabilities.

To install globally:-

Then inside your project directory, where package.json is located, run this:-

Here’s a sample of the vulnerabilities found in just one of our production apps (security vulnerabilities now fixed!).

Node security vulnerability report showing 13 found vulnerabilities.
Node security vulnerability report showing 13 found vulnerabilities.

After upgrading all the affected packages and rerunning nsp check, I now get no security alerts reported:-

Node security vulnerability report showing no known vulnerabilities.
Node security vulnerability report showing no known vulnerabilities. All fixed.

Good times.

Take this further by incorporating nsp into your build processes using gulp to help stay on top of security.

Check out Node Security Project for more information.