HTTP Client with Core Http

HTTP Client with Core http

In web development we often want to make HTTP requests to other services, and Node provides a core module http to make such requests. This module uses the event emitter pattern. The idea is that you get a small chunk of the overall response (usually a single line of the overall payload/body) during each data event. You can process the data right away (preferred for large data) or save it all together in a buffer variable for future use once all the data has been received (preferred for JSON).

Take a look at the example in http-get-no-buff.js in which each new line (chunk) of the response is printed back to the terminal with console.log:

http-get-no-buff.js:

const http = require('http')
const url = 'http://nodeprogram.com'
http.get(url, (response) => {
  response.on('data', (chunk) => { 
    console.log(chunk.toString('utf8'))
  })
  response.on('end', () => {
    console.log('response has ended')
  })
}).on('error', (error) => {
  console.error(`Got error: ${error.message}`)
})

The result of running this script will be the home page HTML from http://nodeprogram.com. It might be hard to notice with a naked eye, but the result will be printed as the request is happening, not all at once in the end of the request.

If you want to wait for the entire response, simply create a new variable as a buffer variable (rawData) and save the chunks (parts of the response, usually lines in the payload/body) into it (rawData).

In http-get.js, all the chunks are saved into a buffer variable named rawData before being logged at the end:

const http = require('http')
const url = 'http://nodeprogram.com'
http.get(url, (response) => {
  let rawData = ''
  response.on('data', (chunk) => { 
    rawData += chunk
  })
  response.on('end', () => {
    console.log(rawData)
  })
}).on('error', (error) => {
  console.error(`Got error: ${error.message}`)
})

The result of running the script http-get.js will be the HTML of http://nodeprogram.com, but the output to the terminal (console.log(rawData)) will happen only at the end of the request when all the data has been received. Again, this might be hard to notice with a naked eye but it'll be obvious when processing large responses over 10Mb in size.

HTTP Client for JSON

To process JSON, developers must get the entire response, otherwise the JSON format won't be valid. For this reason, we use the buffer variable rawData. When the response has ended, we parse the JSON. Take a look at the code in http-json-get.js:

const https = require('https')
const url = 'https://gist.githubusercontent.com/azat-co/a3b93807d89fd5f98ba7829f0557e266/raw/43adc16c256ec52264c2d0bc0251369faf02a3e2/gistfile1.txt'
https.get(url, (response) => {
  let rawData = ''
  response.on('data', (chunk) => { 
    rawData += chunk 
  })
  response.on('end', () => {
    try {
      const parsedData = JSON.parse(rawData)
      console.log(parsedData)
    } catch (e) {
      console.error(e.message)
    }
  })
}).on('error', (error) => {
  console.error(`Got error: ${error.message}`)
})

The result of running this script would be the parsed JSON object. The parsing needs to happen inside of the try/catch to handle any failures that may occur due to mal-formated JSON input.

HTTP client POST request

So far we have been using GET requests to receive data from a server. GET requests are able to be used to receive data but you can not send data with a GET request. In order to send a body of data with a request you must use a POST request. POST requests are generally used to upload data or to send data to be processed and returned.

The http core module methods allow you to specify what type of request you want to make. To do so, first create an options object and set the method attribute to the desired request type (‘POST’, 'GET', etc.). Then, use the options object as the first argument when calling http.request().

The following code from http-post.js uses an options object to specify that it is trying to make a POST request:

const http = require('http')
const postData = JSON.stringify({ foo: 'bar' })

const options = {
  hostname: 'mockbin.com',
  port: 80,
  path: '/request?foo=bar&foo=baz',
  method: 'POST',
  headers: {
    'Content-Type': 'application/x-www-form-urlencoded',
    'Content-Length': Buffer.byteLength(postData)
  }
}

const req = http.request(options, (res) => {
  res.on('data', (chunk) => {
    console.log(`BODY: ${chunk}`)
  })
  res.on('end', () => {
    console.log('No more data in response.')
  })
})

req.on('error', (e) => {
  console.error(`problem with request: ${e.message}`)
})

req.write(postData)
req.end()

As a result, the script will send the data to the server (mockbin.com) in a POST request, and output the response of the request.

Ref: 

1. Microsoft: DEV283x: Intro to Node.js