Easy Web Services with PhantomJS [With 3 Samples]

PhantomJS captures web contents and allows to access and manipulate web pages with the standard DOM API. Therefore, it is an optimal tool for me to automate tasks in sites that do not expose their logic as web services. However, there are times when you need to interact and consume web services with PhantomJS without relying on the standard DOM API. Let’s see how can you consume web Services with PhantomJS.

Why to Consume Web Services with PhantomJS?

For some of those web services, I have already written Node.js modules that consume them. I wrote these modules with the help of the request module. It would be lovely to use those tested modules with phantomJS.

However, PhantomJS is not currently written as a Node.js module since the complexity of the task. Moreover, as every Node.js module is essential “a slave” to Node.js core – You cannot use it as PhantomJS as PhantomJS needs to have complete control over everything: The event loop, the network stack, and the JavaScript execution.

Therefore, I wrote the following PhRequest class and its request method. The request method mimics the semantics of the request module.

Consume Web Services with PhantomJS

The code is written in typescript:

PhRequest class
 export class PhRequest {
     url:string = ''
     urlQuery : string
     qs : any = {}
     headers: any = {}
     query: string
     form : any = {};
     data: string = '';
     addJson(item:any) : void { ... }
     set method(value:string) {
         this.operation = value;
     }
     request(page : any , onDone : (text:string,code:number) => void ) { ... }
 }

The PhRequest class has the following properties:

PropertyDescription
urlbase URL of endpoint
methodHTTP request method. This can be any of GET, POST, PUT and so on
qsobject whose properties are the query string parameters
headersobject whose properties are HTTP headers of the request
urlQueryGenerated URL, in other words it is the base URL with query string
queryGenerated query string from the qs object
dataBody of the request. It will be automatically updated if form is not empty or addJson was called.
formobject whose properties are the POST request, if not empty , data will be urlencoded string of the object and the Content-Type will be set ti application/x-www-form-urlencoded
addJson methodtakes an object as parameter. This object is converted to JSON and assigned the PhRequest.data. The method also set the Content-Type header to application/json
request methodsend the request with the the help of the given WebPage. When the request is done , the onDone function is called with the status code of the response and body text of the response.

The request method trying to mimic the request function from Node.js request module. It simply calls the request method of the given PhRequest object.

The Request Method
 function request( page:any , rq:PhRequest , onDone : (text:string,code:number) => void) {
     rq1.request(page, onDone);
 }

How To Consume Web Services with PhantomJS?

Sample 1: GET Request Without Parameters

In the following sample we send a GET request without parameters:

sample_01.js – GET request
 var page  = require('webpage').create();
 var phreq = require("./phrequest");
 var rq = new phreq.PhRequest();

 rq.url = 'https://jsonplaceholder.typicode.com/posts/1';

 phreq.request(page, rq, function (text, code) {
     console.log('The code is ==> ' + code);
     console.log(text);
     phantom.exit(0);
 });
The Output
 The code is ==> 200
{
"userId": 1,
"id": 1,
"title": "...",
"body": "..."
}

Sample 2: GET Request With Parameters

In the following sample, we send a GET request with parameters. The rq.urlQuery will be set to the actual endpoint URL. sample_02.js –

GET request with Query Parameters
 var page  = require('webpage').create();
 var phreq = require("./phrequest");
 var rq = new phreq.PhRequest();

 var rq = new phreq.PhRequest;
 rq.url = 'https://jsonplaceholder.typicode.com/posts'
 rq.qs.userId = 2;

 phreq.request(page, rq, function (text, code) {
     console.log('The URL is  ==> ' + rq.urlQuery);
     console.log('The code is ==> ' + code);
     console.log(text);
     phantom.exit(0);
 });
The Output
 The URL is  ==> https://jsonplaceholder.typicode.com/posts?userId=2
 The code is ==> 200
 [
   {
     "userId": 2,
     "id": 11,
     "title": "...",
     "body": "..."
   },
   ...
 ]

Sample 3 – POST request with JSON as body

In the following sample we send a POST request with JSON as body. The addJson body takes an object as parameter. This object is converted to JSON and assigned the PhRequest.data. The method also set the Content-Type header to application/json.

POST request with JSON body
 var page  = require('webpage').create();
 var phreq = require("./phrequest");
 var rq = new phreq.PhRequest();

 var rq = new phreq.PhRequest;
 rq.url    = 'https://jsonplaceholder.typicode.com/posts'
 rq.method = 'POST';
 rq.addJson( {title: 'foo bar',body: 'bar',userId: 2} )

 phreq.request(page, rq, function (text, code) {
     console.log('The code is ==> ' + code);
     console.log(text);
     phantom.exit(0);
 });
The Output
 The code is ==> 201
 {
   "title": "foo bar",
   "body": "bar",
   "userId": 2,
   "id": 101
 }

Leave a Reply

Your email address will not be published. Required fields are marked *