PhantomJS captures web contents and allows to access and manipulate web pages with the standard DOM API. Therefore, it is an optimal tool for me to automate tasks in sites that do not expose their logic as web services. However, there are times when you need to interact and consume web services with PhantomJS without relying on the standard DOM API. Let’s see how can you consume web Services with PhantomJS.
Why to Consume Web Services with PhantomJS?
For some of those web services, I have already written Node.js modules that consume them. I wrote these modules with the help of the request module. It would be lovely to use those tested modules with phantomJS.
However, PhantomJS is not currently written as a Node.js module since the complexity of the task. Moreover, as every Node.js module is essential “a slave” to Node.js core – You cannot use it as PhantomJS as PhantomJS needs to have complete control over everything: The event loop, the network stack, and the JavaScript execution.
Therefore, I wrote the following PhRequest
class and its request
method. The request
method mimics the semantics of the request module.
Consume Web Services with PhantomJS
The code is written in typescript:
PhRequest class
export class PhRequest {
url:string = ''
urlQuery : string
qs : any = {}
headers: any = {}
query: string
form : any = {};
data: string = '';
addJson(item:any) : void { ... }
set method(value:string) {
this.operation = value;
}
request(page : any , onDone : (text:string,code:number) => void ) { ... }
}
The PhRequest
class has the following properties:
Property | Description |
---|---|
url | base URL of endpoint |
method | HTTP request method. This can be any of GET, POST, PUT and so on |
qs | object whose properties are the query string parameters |
headers | object whose properties are HTTP headers of the request |
urlQuery | Generated URL, in other words it is the base URL with query string |
query | Generated query string from the qs object |
data | Body of the request. It will be automatically updated if form is not empty or addJson was called. |
form | object whose properties are the POST request, if not empty , data will be urlencoded string of the object and the Content-Type will be set ti application/x-www-form-urlencoded |
addJson method | takes an object as parameter. This object is converted to JSON and assigned the PhRequest.data. The method also set the Content-Type header to application/json |
request method | send the request with the the help of the given WebPage . When the request is done , the onDone function is called with the status code of the response and body text of the response. |
The request
method trying to mimic the request function from Node.js request module. It simply calls the request method of the given PhRequest object.
The Request Method
function request( page:any , rq:PhRequest , onDone : (text:string,code:number) => void) {
rq1.request(page, onDone);
}
How To Consume Web Services with PhantomJS?
Sample 1: GET Request Without Parameters
In the following sample we send a GET request without parameters:
sample_01.js – GET request
var page = require('webpage').create();
var phreq = require("./phrequest");
var rq = new phreq.PhRequest();
rq.url = 'https://jsonplaceholder.typicode.com/posts/1';
phreq.request(page, rq, function (text, code) {
console.log('The code is ==> ' + code);
console.log(text);
phantom.exit(0);
});
The Output
The code is ==> 200
{
"userId": 1,
"id": 1,
"title": "...",
"body": "..."
}
Sample 2: GET Request With Parameters
In the following sample, we send a GET request with parameters. The rq.urlQuery will be set to the actual endpoint URL. sample_02.js –
GET request with Query Parameters
var page = require('webpage').create();
var phreq = require("./phrequest");
var rq = new phreq.PhRequest();
var rq = new phreq.PhRequest;
rq.url = 'https://jsonplaceholder.typicode.com/posts'
rq.qs.userId = 2;
phreq.request(page, rq, function (text, code) {
console.log('The URL is ==> ' + rq.urlQuery);
console.log('The code is ==> ' + code);
console.log(text);
phantom.exit(0);
});
The Output
The URL is ==> https://jsonplaceholder.typicode.com/posts?userId=2
The code is ==> 200
[
{
"userId": 2,
"id": 11,
"title": "...",
"body": "..."
},
...
]
Sample 3 – POST request with JSON as body
In the following sample we send a POST request with JSON as body. The addJson body takes an object as parameter. This object is converted to JSON and assigned the PhRequest.data. The method also set the Content-Type
header to application/json
.
POST request with JSON body
var page = require('webpage').create();
var phreq = require("./phrequest");
var rq = new phreq.PhRequest();
var rq = new phreq.PhRequest;
rq.url = 'https://jsonplaceholder.typicode.com/posts'
rq.method = 'POST';
rq.addJson( {title: 'foo bar',body: 'bar',userId: 2} )
phreq.request(page, rq, function (text, code) {
console.log('The code is ==> ' + code);
console.log(text);
phantom.exit(0);
});
The Output
The code is ==> 201
{
"title": "foo bar",
"body": "bar",
"userId": 2,
"id": 101
}