 |
Compares prices on a product by using REST robots that searches Froogle, EBay and Yahoo Shopping. The demo uses JSP to call the robots, parse the returned XML, sort the result and present the result.his document will explain how the
robots work and how to use openkapow REST robots from JSP. Robots and
source code are available for download and can work as a basis to build
other Java-based mashups. |
   |
The Demo
In this demo three different sites will be searched by threee
different REST robots and then the result will be sorted and presented
to the user using JSP. The process of
the demo is:
- The user enters the name of the product that he/she wants
to compare the price for
- In the JSP three different robots (towards Froogle, Ebay
and
Yahoo Shopping) are executed. The returned XML is parsed and the
results are sorted according to price.
- The result is shown to the user
The REST robots
The three REST robots this demo uses all have the same input and
output. The input parameters are
searchText
and
numberOfItems. The
first one is the text the robots will use to search in the web sites
and the robot will return a maximum of
numberOfItems
items. The output
from all robots is XML where each search result is in the format:
<item>
<title>..</title>
<price>..</price>
<url>..</url>
<source>..</source>
</item>
Title,
price and
URL
are all related to a specific search result.
Source is the name of the site searched (eg
Ebay, Froogle or Yahoo
Shoppping) plus the name of the store which the price acctually comes
from (this since Froogle and Yahoo are price search aggregators in
themselves), for example "Froogle: Circuit City". All robots loop
through several pages of search results if needed. Each of the robots
are only set to return a maximum of 10 prices each, just for demo
purposes.
Putting it all together in JSP
In this demo all REST robots are called in the method
getPrices
that
takes the URL to the robot as well as a
TreeSet
to store the prices in
as input parameters. This method is called three times (once each for
Froogle, Ebay and Yahoo Shopping) and then the content of the
TreeSet
is displayed to the user. The content of the
TreeSet
(ie all prices
found) are sorted with the
ComparePrices
Comparator class as prices are
added to the
TreeSet.
Calling REST robots from Java
The openkapow REST robots are called via standard HTTP GET requests. So
it is easy to call the robots from Java using
HttpURLConnection
with a
java.net.URL object created with the REST
robots URL. The XML the robot
returns can be accessed in the
java.io.InputStream
of the
HttpURLConnection. When the XML as been read
the
HttpURLConnection of
course have to be closed.
HttpURLConnection conn = null;
...
URL url = new URL(urlStr);
conn =
(HttpURLConnection)url.openConnection();
...
doc = builder.parse(conn.getInputStream());
...
conn.disconnect();
Parameters are sent to the REST robots as GET parameters in the URL used to call them. This is an example of how such a URL is assembled in the Java code:
http://service.openkapow.com/demo/EbayREST.rest?numberOfItems=10&resultformat=xml
&searchText="+URLEncoder.encode(request.getParameter("searchText"))
The GET parameters specify that 10 items should be returned for the search of searchText and that the output from the robot should be formated as XML.
Parsing XML in Java
The REST robots in this demo all return XML formated in the same way.
So we can parse the output from all three robots in the same way. First
of all a new instance of
javax.xml.parsers.DocumentBuilderFactory is
needed, using this instance
a new
javax.xml.parsers.DocumentBuilder is
created. This
DocumentBuilder is used to parse the
InputStream
from the
HttpURLConnection and put the XML into a
org.w3c.dom.Document.
DocumentBuilderFactory docFactory =
DocumentBuilderFactory.newInstance();
docFactory.setValidating(false);
docFactory.setNamespaceAware(false);
docFactory.setIgnoringElementContentWhitespace(false);
DocumentBuilder builder = null;
builder = docFactory.newDocumentBuilder();
Document doc = null;
doc = builder.parse(conn.getInputStream());
Once the XML is in the
Document object the
demo loops through all the
<item> tags in the XML. For each
of the
<item> tags
title,
price,
url and
source
are extracted
from the XML and stored in a
HashMap object.
This
HashMap is then added
to the
TreeSet that contains all prices.
NodeList itemList = doc.getElementsByTagName("item");
for (int i=0; i<itemList.getLength();i++){
Node item = itemList.item(i);
Node tag = item.getFirstChild();
HashMap map = new HashMap();
while (tag !=null){
if
(tag.getNodeType() == Node.ELEMENT_NODE){
if
(tag.getNodeName().equalsIgnoreCase("title"))
map.put("title", tag.getFirstChild().getNodeValue());
else if
(tag.getNodeName().equalsIgnoreCase("price"))
map.put("price", tag.getFirstChild().getNodeValue());
else if
(tag.getNodeName().equalsIgnoreCase("url"))
map.put("url", tag.getFirstChild().getNodeValue());
else if
(tag.getNodeName().equalsIgnoreCase("source"))
map.put("source", tag.getFirstChild().getNodeValue());
}
tag =
tag.getNextSibling();
}
result.add(map);
}
Summary
This demo shows how to use openkapow REST robots from Java/JSP, both
how to call them and how to parse the XML returned. With just three
robots and a few lines of Java code an aggregated price search page has
been built. It can easily be expanded to search more sources (just add
some more robots with the same XML output) or this demo can be used as
an example to build mashups using several sources for functionality and
data. In this case the sources were Froogle, Ebay and Yahoo Shopping,
but in your own mashups it might be any sources that you can access
over the web.