The motive

PhantomJS allows to capture web contents and to access and manipulate web pages with the standard DOM API. This makes it optimal tool for me for automating tasks in sites which do not expose their logic as web services. However, there are times when I need to interact with a web service while using PhantomJS.

For some of those web services, I have already write Node.js module which consume them. Those modules was written with the help of the request module. It would be lovely to use those tested modules with phantomJS.

However, PhantomJS is not currently written as Node.js module since the complexity of the task. As every Node.js module is essentially “a slave” to Node.js core - It can not be used in PhantomJS as PhantomJS needs to have full control over everything: The event loop, the network stack, and the JavaScript execution.

This is motive to write PhRequest class and request method, which will mimic the semantics of the request module.

The PhRequest class and request function

The code is written in typescript:

PhRequest class
export class PhRequest {
    url:string = ''
    urlQuery : string
    qs : any = {}
    headers: any = {}
    query: string
    form : any = {};
    data: string = '';
    addJson(item:any) : void { ... }
    set method(value:string) {
        this.operation = value;
    }
    request(page : any , onDone : (text:string,code:number) => void ) { ... }
}

The PhRequest class has the following properties:

PropertyDescription
urlbase URL of endpoint
methodHTTP request method. This can be any of GET, POST, PUT and so on
qsobject whose properties are the query string parameters
headersobject whose properties are HTTP headers of the request
urlQueryGenerated URL, in other words it is the base URL with query string
queryGenerated query string from the qs object
dataBody of the request. It will be automatically updated if form is not empty or addJson was called.
formobject whose properties are the POST request, if not empty , data will be urlencoded string of the object and the Content-Type will be set ti application/x-www-form-urlencoded
addJson methodtakes an object as parameter. This object is converted to JSON and assigned the PhRequest.data. The method also set the Content-Type header to application/json
request methodsend the request with the the help of the given WebPage. When the request is done , the onDone function is called with the status code of the response and body text of the response.

The request method trying to mimic the request function from Node.js request module. It simply call the request method of the given PhRequest object.

request function
function request( page:any , rq:PhRequest , onDone : (text:string,code:number) => void) {
    rq1.request(page, onDone);
}

Usage

In the following sample we send a GET request without parameters:

samle_01.js - GET request
var page  = require('webpage').create();
var phreq = require("./phrequest");
var rq = new phreq.PhRequest();
rq.url = 'https://jsonplaceholder.typicode.com/posts/1';
phreq.request(page, rq, function (text, code) {
    console.log('The code is ==> ' + code);
    console.log(text);
    phantom.exit(0);
});
samle_01.js - Output
The code is ==> 200
{
  "userId": 1,
  "id": 1,
  "title": "...",
  "body": "..."
}

In the following sample we send a GET request with parameters. The rq.urlQuery will be set to the actual endpoint URL.

sample_02.js - GET request with Query Parameters
var page  = require('webpage').create();
var phreq = require("./phrequest");
var rq = new phreq.PhRequest();
var rq = new phreq.PhRequest;
rq.url = 'https://jsonplaceholder.typicode.com/posts'
rq.qs.userId = 2;
phreq.request(page, rq, function (text, code) {
    console.log('The URL is  ==> ' + rq.urlQuery);
    console.log('The code is ==> ' + code);
    console.log(text);
    phantom.exit(0);
});
sample_02.js - output
The URL is  ==> https://jsonplaceholder.typicode.com/posts?userId=2
The code is ==> 200
[
  {
    "userId": 2,
    "id": 11,
    "title": "...",
    "body": "..."
  },
  ...
]

In the following sample we send a POST request with JSON as body. The addJson body takes an object as parameter. This object is converted to JSON and assigned the PhRequest.data. The method also set the Content-Type header to application/json.

sample_03.js - POST request with JSON body
var page  = require('webpage').create();
var phreq = require("./phrequest");
var rq = new phreq.PhRequest();
var rq = new phreq.PhRequest;
rq.url    = 'https://jsonplaceholder.typicode.com/posts'
rq.method = 'POST';
rq.addJson( {title: 'foo bar',body: 'bar',userId: 2} )
phreq.request(page, rq, function (text, code) {
    console.log('The code is ==> ' + code);
    console.log(text);
    phantom.exit(0);
});
sample_03.js - output
The code is ==> 201
{
  "title": "foo bar",
  "body": "bar",
  "userId": 2,
  "id": 101
}