Week 11
Production Preparation

As you work on completing the final project, it is time to look at some techniques to better manage environment specific application configuration and security settings.

# Agenda

• AMA (10 mins)
• Quiz (10 mins)
• Logging (20 mins)
• Configuration Settings (20 mins)
• Compression (2 mins)
• Security Middleware (6 mins)
• NPM Audit (2 mins)
• Lab Time

# AMA / Review

# Validate ID

Our earlier solution (from week 8) to checking for a valid ID was incomplete and will produce two different error messages. I was hoping that you would discover this through your testing for the A3 assignment, and some of you did.

If the _id value is correctly formatted but does not exist in the database we will get the desired ResourceNotFound error.

However, if in your testing you used a simple integer value like 5 for the ID, Mongoose will respond with a type cast error instead. It would be preferable to return the same ResourceNotFound error in either case.

To check if the user supplied id is in the correct ObjectId format, we use mongoose.Types.ObjectId.isValid(id). I use a small helper function for this ...

// Helper functions
const validateId = async id => {
  if (mongoose.Types.ObjectId.isValid(id)) {
    if (await Student.countDocuments({_id: id})) return true
  }
  throw new ResourceNotFound(`Could not find a Student with id ${id}`)
}

... and then in the route handler simply call await validateId(req.params.id).

# Logging

In production applications, it is a good practice to use a logger like Winston instead of writing debug or console.log statements. Instead of just writing debug statements to the console, a logger can also write them out to a file or database table that can be analysed later.

A logger lets you tag your output with a severity level. This makes it easier to filter for right level of detail.

# Winston

Let's install and setup Winston.

npm install winston

Now define a new logger.js module in the /startup folder.

const winston = require('winston')
const logger = winston.createLogger({
  level: 'info',
  format: winston.format.json(),
  defaultMeta: {service: 'user-service'},
  transports: [
    //
    // - Write to all logs with level `info` and below to `combined.log`
    // - Write all logs error (and below) to `error.log`.
    //
    new winston.transports.File({filename: 'error.log', level: 'error'}),
    new winston.transports.File({filename: 'combined.log'})
  ]
})

//
// If we're not in production then log to the `console` with the format:
// `${info.level}: ${info.message} JSON.stringify({ ...rest }) `
//
if (process.env.NODE_ENV !== 'production') {
  logger.add(
    new winston.transports.Console({
      format: winston.format.simple()
    })
  )
}

module.exports = logger

Import this logger module in every other module and update any console.log or debug statements. For example in the /startup/connectDatabase module ...

const logger = require('./logger')
module.exports = () => {
  const mongoose = require('mongoose')
  mongoose
    .connect(`mongodb://localhost:27017/mad9124`, {
      useNewUrlParser: true,
      useUnifiedTopology: true,
      useCreateIndex: true,
      useFindAndModify: false
    })
    .then(() => {
      logger.log('info', `Connected to MongoDB ...`)
    })
    .catch(err => {
      logger.log('error', `Error connecting to MongoDB ...`, err)
      process.exit(1)
    })
}

Winston is a very flexible logger with a lot of additional capabilities. Be sure to read the full documentation to know what it can do.

# NODE_ENV

Like in the Winston configuration above, it is very common to have conditional logic in the main app startup module(s) that set different behaviours based on the deployment environment. For example, we generally want to see debugging or detailed logging during development, but want to suppress these in production.

As you know we can access command shell environment variables using the process.env object. It is a common best practice to set the NODE_ENV environment variable to indicate the current runtime environment. Some typical values are: dev, development, test, staging, prod, production.

If NODE_ENV is not set, process.env.NODE_ENV returns undefined.

Because this is such a common use case, Express provides another way to read the NODE_ENV environment variable – using app.get('env'). If NODE_ENV is not set, app.get('env') returns development as a default value.

To declare the production environment run this command in the terminal.

export NODE_ENV=production

To set it back to development simply reset the value of the NODE_ENV variable.

export NODE_ENV=development

# Configuration Settings

There are many configuration parameters for an application, such as database connection details, encryption keys, third party API access keys, etc. Some or all of these details may change from one environment to the next.

To make our code less brittle, we centralize most these parameters in some JSON files while others should not be stored in source control and are supplied through environment variables.

To simplify accessing these configuration parameters throughout our application code, we can use an NPM package simply called config.

npm install config
mkdir config
touch config/default.json

Edit the /config/default.json file to add any parameters that we want to be able to access elsewhere in our code.

{
  "name": "My Application",
  "db": {
    "host": "localhost",
    "port": "27017",
    "dbName": "mad9124-mckennr"
  },
  "jwt": {
    "saltRounds": 14
  }
}

We can access these parameters in other code modules by requiring the config module and then using it's config.get() method. For example we could re-write the /startup/databaseConnect module like this.

const config = require('config')
const logger = require('./logger')

module.exports = () => {
  const mongoose = require('mongoose')
  const dbConfig = config.get('db')
  mongoose
    .connect(`mongodb://${dbConfig.host}:${dbConfig.port}/${dbConfig.dbName}`, {
      useNewUrlParser: true,
      useUnifiedTopology: true,
      useCreateIndex: true,
      useFindAndModify: false
    })
    .then(() => {
      logger.log('info', `Connected to MongoDB ...`)
    })
    .catch(err => {
      logger.log('error', `Error connecting to MongoDB ...`, err)
      process.exit(1)
    })
}

We can additional files named to match the NODE_ENV setting. For example to override the hostname of the database server in production, we can create a production.json file in the /config folder.

{
  "db": {
    "host": "db.myapp.io"
  }
}

The config module will merge the properties in default.json with those of the current environment as set in NODE_ENV.

# Environment Variables

Some deployment situations rely heavily on environment variables to configure secrets and settings best left out of a codebase.

We can map environment variables to the config settings object. Create a new file called custom-environment-variables.json in the config folder.

{
  "db": {
    "userName": "APP_DBUSER",
    "password": "APP_DBPASSWORD"
  },
  "jwt": {
    "secretKey": "APP_JWTKEY"
  }
}

In the terminal, set the values for these environment variables.

export APP_DBUSER=datagod
export APP_DBPASSWORD=supersecret
export APP_JWTKEY=myOtherSuperSecretKey

Then in our code we can access these properties from the config object as normal.

console.log(config.get(db.userName)) // will print datagod

# Generate JWT Secret

The JWT secret key can be any string value. To make it more effective, it should be at least 30 characters and preferably random. Here is a simple script to generate a new random key. Create a new top level file called genKey.js

console.log([...Array(30)].map(e => ((Math.random() * 36) | 0).toString(36)).join(''))

When you are setting up your deployment environment, run the script and then copy the output to set the environment variable.

node genKey.js
ixzz7ph7goovu62b6hz3k6egyghhbn

export APP_JWTKEY=ixzz7ph7goovu62b6hz3k6egyghhbn

# Compression

To improve network communications performance, we look for any opportunity to reduce the payload size for any given request/response cycle. One such possibility is to use standard text compression algorithms on the response payload.

The NPM module called compression is a Node.js compression middleware. It will attempt to compress the response.body using gzip for responses with a compatible Content-Type header value (e.g. text/html or application/json). See the compressible module for default behaviour.

npm install compression

const compression = require('compression')
const express = require('express')
const app = express()

// attempt to compress all routes
app.use(compression())

# Security Middleware

As we have discussed before, good application security is not "a feature" and not "a bolt-on module". It is about applying multi-layered best practices throughout the design and development cycle. As we prepare to deploy our web service to a production environment, CORS and Helmet are two important middleware modules that help.

# CORS

Read this HTML5 Rocks backgrounder on Cross-Origin Resource Sharing (CORS).

The use-case for CORS is simple. Imagine the site alice.com has some data that the site bob.com wants to access. This type of request traditionally wouldn’t be allowed under the browser’s same origin policy. However, by supporting CORS requests, alice.com can add a few special response headers that allows bob.com to access the data.

To manage CORS in our Express web service use the cors package from NPM.

npm install cors

Then require it in the main app.js module and apply it as the first middleware with the default settings.

const express = require('express')
const cors = require('cors')

const app = express()

app.use(cors())
// other middleware goes here

The cors() middleware constructor function takes an optional configuration options object. The default settings are equivalent to setting the options object with these values.

{
  "origin": "*",
  "methods": "GET,HEAD,PUT,PATCH,POST,DELETE",
  "preflightContinue": false,
  "optionsSuccessStatus": 204
}

This will let a client app served from any domain name access the API resources.

# Helmet

The helmet NPM module is an integrated middleware package (with 14 sub-modules) that implements many network communications security best practices.

It's not a silver bullet, but it can help!

npm install helmet

const express = require('express')
const helmet = require('helmet')

const app = express()

app.use(helmet())

Not all of the included middleware functions are enabled by default. See the full documentation for all of the options and details about the kinds of attacks they help to prevent.

# NPM audit

The NPM package manager has a built-in function to scan the entire dependency tree for known security vulnerabilities. This will output a list of packages that may need to be updated or replaced with a more secure library module. It is a good idea to run this check regularly, as new attack vectors are discovered and reported on an ongoing basis.

npm audit

# For next week

Before next week's class, please read these additional online resources.

Martin Fowler's Serverless Architectures