build with purpose

Effective Endpoint Design Techniques

Maintainable, Testable, Performant, and that's what I like about it

I’ve recently come across some pretty effective concepts surrounding the design of endpoints. My career has been focused on backend server code recently where I’ve needed to create, extend, and refactor endpoints. I’d now like to share what I’ve discovered along the way that has made my endpoint code more maintainable, testable, and performant.

I’ll be using a little bit of ES6 style NodeJS code with Express syntax to explain my points. There’s also a little generic SQL code in there as well. I think it should be easy to follow along even if you aren’t familiar with all of those technologies.

Let’s start with a simple endpoint to get the balance of an account:

// accountResource.js
const nodeSQL = require('node-sql')
// ...
// GET endpoint definition
router.get('accountBalance', (request, response) => {
  if (!request.accountId) {
    return response.status(400).send('missing accountId')
  }
  let sql = 'SELECT balance FROM Accounts WHERE ID = ${request.accountId}'
  nodeSQL.exec(sql, CONFIG, (error, result) => {
    if (error) return response.sendStatus(500)
    response.status(200).json(result.balance)
  })
})

This is what most endpoints start out looking like when you’re just trying to get something working. All of your logic is in one method and everything is fairly simple. This should work fine for a while and will definitely work in production. However, there’s a lot of low hanging fruit to improve when looking at this code from a testing and maintenance standpoint.

The Data Access Layer

The first improvement that I’d like to make is to pull the database code out into a Data Access Layer or Database Access Object (DAO). This is best practice for accessing databases because having database code in a separate DAO moves all of your database logic to one place. It then becomes easy to see how you communicate to the database across all requests and it’s easier to reuse existing request methods. Also, having a DAO helps with testing because it’s easier to mock the results coming back from a DAO function rather than having to mock the database response in every test.

Second, the SQL statement should not be an in-line statement and should instead be a stored procedure that is called. Having a stored procedure allows programmers and database administrators to change production SQL scripts without having to release the project. It also separates the implementation of your SQL queries from how the data they produce is used.

As a final point, we should probably also check the accountId to make sure that the request doesn’t try to inject SQL scripts. This is a major security concern and could open up control of your database to an attacker. Fortunately, many node libraries have a section related to SQL injection.

Let’s go ahead and pull out a separate DAO to request that data:

// accountDao.js
export function getBalance(id) {
  return new Promise((resolve, reject) => {
    const request = new sql.Request()
    request.input('id', sql.Int, '--')
    request.query('GetAccountBalance', (err, result) => {
      if (err) return reject()
      resolve(result.balance)
    })
  })
}

Now we’ve got a separate function in our DAO and we’re returning a promise to inform the caller if the query was successful. Now all we need to do is hook that up to our endpoint logic to return the request.

// accountResource.js
const accountDao = require('./accountDao')
// ...
router.get('accountBalance', (request, response) => {
  if (!request.accountId) {
    return response.status(400).send('missing accountId')
  }
  accountDao.getBalance(request.accountId)
    .then((result) => {
      return response.status(200).json(result)
    }, () => {
      return response.sendStatus(500)
    })
})

This is a much more maintainable and testable design. We can now test the DAO and the resource separately and write integration tests to ensure everything works correctly. When I think of simple requests to a database where data requests are made simply without much transformation, this setup is what I have in mind. At this point, I’d be done improving such an endpoint.

The Service Layer

Now, a little bit of time passes and this solution doesn’t meet the business needs anymore. Balance is no-longer a variable that can be found in the Accounts table and instead needs to be calculated from the Transactions table. To do this we need to change our DAO a little bit:

// accountDao.js
export function getBalance(id) {
  return new Promise((resolve, reject) => {
    const request = new sql.Request()
    request.input('accountId', sql.Int, '--')
    request.query('GetTransactionHistory', (err, results) => {
      if (err) return reject()
      let balance = results.map(t => t.amount).reduce((a, b) => a + b)
      resolve(balance)
    })
  })
}

We’ve added the new stored procedure GetTransactionHistory and we’re calculating the summation of the resulting amounts from the transactions. It’s a simple change and if we’re in a hurry we might just leave this here. However, we might want to use this summation function on other result sets in the future and we also would benefit from breaking this functionality out into a separate unit test.

I propose introducing a service layer to handle getting balance and handling the summation of the results. The service layer will be the layer the resource will communicate with and it will orchestrate transforming data and making requests to the DAO.

// accountService.js
const accountDao = require('./accountDao')
// ...
let calculateBalance = (transactions) => {
  return transactions.map(t => t.amount).reduce((a, b) => a + b)
}

export function getBalance(id) {
  return new Promise((resolve, reject) => {
    accountDao.getBalance(req.accountId)
      .then((result) => {
        resolve(calculateBalance(result))
      }, () => {
        reject()
      })
  })
}
// accountDao.js
export function getBalance(id) {
  return new Promise((resolve, reject) => {
    const request = new sql.Request()
    request.input('accountId', sql.Int, '--')
    request.query('GetTransactionHistory', (err, results) => {
      if (err) return reject()
      resolve(res)
    })
  })
}

The accountResource also needs to change to call the new accountService.

Any future changes in the underlying logic should happen in the service layer independent of resources and DAOs. The resource layer now only deals with validating the request and persisting the reponse. The DAO is only concerned with making database requests and returning the result.

The Context Layer

Now, the business needs change again and some accounts have different types of transactions. These transactions can be found in a different database in another Transactions table. The DAO could know about two different databases, but I’d prefer to have two separate DAOs (once for each database for simplicity). However, the issue is that this logic of which DAO to request gets moved up to the service layer. This isn’t the right place to figure out which DAO to request. Instead, I propose that we introduce a context layer.

The context layer helps simplify requesting data from DAOs. This layer should be responsible for gathering up all the data from DAOs and formatting it so that the service layer can interpret the data. Currently our implementation of the service layer assumes that the data will come in as a list of objects with an amount attribute. If this isn’t the case, we need to format the data appropriately for the service to be able to process the data.

Let’s go ahead and setup a context layer to request from two different daos depending on the type of account.

First, lets specify the new transactionDao that will request our new type of transactions:

// transactionDao.js
export function getBalance(id) {
  return new Promise((resolve, reject) => {
    const request = new sql.Request()
    request.input('accountId', sql.Int, '--')
    request.query('GetBTypeTransactionHistory', (err, results) => {
      if (err) return reject()
      resolve(results)
    })
  })
}

Now, we need to add a way to get the type of the requested account. Let’s go ahead and add that to the accountDao:

// accountDao.js
//...
export function getAccountType(id) {
  return new Promise((resolve, reject) => {
    const request = new sql.Request()
    request.input('id', sql.Int, '--')
    request.query('GetAccountType', (err, results) => {
      if (err) return reject()
      resolve(results.type)
    })
  })
}

Finally let’s implement the context layer and handle requesting from each DAO depending on the type of account:

// accountContext.js
const accountDao = require('./accountDao')
const transactionDao = require('./transactionDao')
// ...
export function getBalance(accountId) {
  return new Promise((resolve, reject) => {
    accountDao.getAccountType(accountId)
      .then((type) => {
        if (type === 'A') {
          accountDao.getBalance(accountId)
            .then((result) => {
              resolve(result)
            }, () => {
              reject()
            })
        } else {
          transactionDao.getBalance(accountId)
            .then((result) => {
              resolve(result)
            }, () => {
              reject()
            })
        }
      })
  })
}

I haven’t seen context layers used widely across the teams I’ve been on, but I’ve found them invaluable when migrating databases and writing tests. It’s also nice to have a place to parse and format the results of database requests.

Final Thoughts

I’ll be updating this post regularly if I come across new concepts or techniques that help me to increase the quality of my endpoints. If you feel like I missed something or if these techniques helped you out I’d like to hear about it. Comment below or send me a message on Twitter.

This definitely isn’t comprehensive of all considerations and possibilities that can come up during endpoint design. However, I’ve found these tools and concepts helpful when writing testable endpoints that need to be performant and maintainable.

Make my day and share this post:

Other posts to peak your interest:

  • Understanding Visual Testing
  • The Redux Saga Black Box
  • What my college degree gave me
  • Technical Leaders Enabling Stronger Teams
  • Finding Your Gateway to Learning Vue
  • Why Write Server Rendered Frontend Apps
  • comments powered by Disqus
    © 2019. All rights reserved.