Easy Ajax with jQuery

Posted in Software on September 4th, 2007 by Valeriu Buzila / No Comments »

Ajax is changing web applications, giving them a responsiveness that’s unheard of beyond the desktop. But behind all the hype, there’s not much to Ajax — (X)HTML, JavaScript, and XML are nothing new, and in this tutorial, I’ll show you how to simplify the process of adding Ajax to your application even further with the help of jQuery, a popular JavaScript library.

What’s Ajax?

You’ve probably heard about Ajax before, or at least used an Ajax-based application — Gmail, for instance. Quite simply, Ajax is a technique for handling external data through JavaScript asynchronously, without reloading the entire page. SitePoint offers a good introduction to Ajax. Jesse James Garrett is credited with coining the term in this article.

Unfortunately, in-depth tutorials on practical ways to enter the world of Ajax are few and far between. To add to the problem, the XMLHttpRequest class used by Ajax isn’t very easy for beginning web developers to use. Luckily, a number of JavaScript libraries offer an easier way. Today I’ll show you how jQuery — one of these libraries — allows you to easily add Ajax to your application.

What’s jQuery?

jQuery is another mature JavaScript library that offers some features that the others do not. Admittedly, it’s not exactly as lightweight as some of the other offerings: jQuery comes in at 19kb, while moo.fx is only 3kb. You can read more about jQuery at The JavaScript Library World Cup for a comparison of a few other JavaScript libraries that offer similar functionality.

Assumed Knowledge

To complete this tutorial, you’ll need some basic JavaScript knowledge. If you know any C-style languages, you’ll get the hang of JavaScript in no time. Just think curly braces, function declarations, and optional semicolons at the end of each line (they’re not optional with jQuery, though). If you’re keen to get started with JavaScript, see this excellent, concise JavaScript tutorial designed for programmers. Also, since we’re talking about web applications, a basic knowledge of HTML is required.

jQuery 101

Let’s walk through a quick introduction to jQuery. To be able to use it in your pages, you’ll first need to download the library. You can download the latest version — 1.1.2 at the time of writing. jQuery’s methodology is simple: find things, do stuff. We select elements from the document (via the DOM) using the jQuery function, aliased as $(). This handy function acts just like document.getElementById(), except that instead of only supporting IDs, it supports CSS selectors and some XPath selectors; and, instead of returning one element, it can return an array of elements. Okay, so maybe a better description of $() is that it’s like document.getElementById() on steroids.

We then use functions to perform actions on our selections. For example, to append the text “Hello World!” to all divs with the class 'foo', then set the color to red, we’d use the following code:

$("div.foo").append("Hello World!").css("color","red");

Easy! Normally, this task would require two lines of code, like so:

$("div.foo").append("Hello World!");
$("div.foo").css("color","red");

jQuery’s chainable methods allow us to write much more compact code than other JavaScript libraries. There are functions in jQuery that don’t need an object, as they work independently, and many of the Ajax functions fall into this group. For example, the post function, which we will soon make use of, is called by typing $.post(parameters). For more jQuery functions, check the online documentation or visualjquery.com.

Example 1 - Our First Ajax Application

As an example, we’re going to make an interactive concept generator. Basically, this involves our selecting two terms at random from a list, then combining them to create a phrase. For this exercise, we’ll use web 2.0 buzzwords (’Mashup’, ‘Folksonomy’, ‘Media’ and so on), and normally we’d fetch these terms from a flat file. To save you from downloading every single combination (or at least every element) in JavaScript, we’re going to generate it on the fly at the server end, and fetch it for the client with jQuery. jQuery integrates perfectly with normal JavaScript, so you’ll find it an easy task to work it into your code.

Server-side Code (PHP)

To keep it simple, we’ll use the basic code below to create our concept generator. Don’t worry about how it works, just look at what it does: it outputs a randomised quote. Note that this code doesn’t output XML — it merely outputs raw text:

<?php
header("Cache-Control: no-cache”);
// Ideally, you’d put these in a text file or a database.
// Put an entry on each line of ‘a.txt’ and use $prefixes = file(”a.txt”);
// You can do the same with a separate file for $suffixes.
$prefixes = array(’Mashup’,'2.0′,’Tagging’,'Folksonomy’);
$suffixes = array(’Web’,'Push’,'Media’,’GUI‘);
// This selects a random element of each array on the fly
echo $prefixes[rand(0,count($prefixes)-1)] . ” is the new ”
. $suffixes[rand(0,count($prefixes)-1)];
// Example output: Tagging is the new Media
?>

Here, I’ve used the Cache-Control header response because Internet Explorer has a habit of caching pages that have the same URL, even if the content between the pages differs. Obviously, that defeats the purpose of our script — the production of a new quote on every load. We could have used jQuery to include a random number in the URL that would then be discarded, but it’s easier to address this caching issue on the server side than the client side.

Client-side Code (HTML)

Let’s start creating the HTML for the front end, then work our Ajax into it. All we need on the page is a button that users can click to request another quote, and a div into which we’ll put the quote once we’ve received it from the server. We’ll use jQuery to select this div and load the quote into it, and we’ll reference the div by its id. If we wanted to, we could use jQuery to load the quote into multiple elements, with the help of a class, but an id is all we need for now. Let’s make this the content of our body element:

<input type="submit" id="generate" value="Generate!">
<div id="quote"></div>

We can put the quote itself inside the div. Normally, we’d have a lengthy onSubmit event for the button (the input with the id 'generate'). Sometimes, we’d have an onSubmit event handler that called a JavaScript function. But with jQuery, we don’t even need to touch the HTML — we can separate behaviour (the event handler) from the structure (the page HTML) with ease.

Client-side Code (jQuery)

It’s time to bring our back end together with our front end using jQuery. I mentioned earlier that we can select elements from the DOM with jQuery. First, we have to select the button and assign an onClick event handler to it. Within the code for this event, we can select the div and load the content of our script into it. Here’s the syntax for the click event handler:

$("element expression").click(function(){
// Code goes here
});

As you probably already know, if we were to select this element in CSS, the # would identify that we were making our selection using the element’s id attribute. You can use exactly the same syntax with jQuery. Therefore, to select the button with the id 'generate' (which we gave it above), we can use the element expression #generate. Also, be aware that this syntax defines our event handler as an anonymous function within the event itself.

Mark Wubben’s JavaScript Terminology page offers a great explanation of anonymous functions, if you’d like to know more.

We’re going to use one of jQuery’s higher level Ajax functions, load(). Let’s assume that our generator script is saved as script.php. Let’s integrate it with our client side with the help of the load() function:

$("#generate").click(function(){
$("#quote").load("script.php");
});

That’s it: three lines of code, and we have fully functioning Ajax random quote generator! Well, almost.

The problem with JavaScript is that code that’s not within a function is executed as soon as the browser reaches it during rendering — not once the page has finished rendering. As such, this code will try to attach to an element that has not yet loaded. Normally, we’d use window.onload to deal with this issue. However, the limitation with that approach is that window.onload is called once everything has finished loading — images and all. We’re not interested in waiting for those images — it’s just the DOM that we want access to.

Fortunately, jQuery has $(document).ready(), which, as its name suggests, is executed when the DOM is ready to be manipulated.

The Complete Code

Here’s the complete code, including the $(document).ready wrapper and some basic HTML and CSS:

<html>
<head>
<title>Ajax with jQuery Example</title>
<script type="text/JavaScript" src="jquery.js"></script>
<script type="text/JavaScript">
$(document).ready(function(){
$("#generate").click(function(){
$("#quote p").load("script.php");
});
});
</script>
<style type="text/css">
#wrapper {
width: 240px;
height: 80px;
margin: auto;
padding: 10px;
margin-top: 10px;
border: 1px solid black;
text-align: center;
}
</style>
</head>
<body>
<div id="wrapper">
<div id="quote"><p> </p></div>
<input type="submit" id="generate" value="Generate!">
</div>
</body>
</html>

This code is also included in this downloadable zip file. Remember, this code assumes the jQuery library has been saved as jquery.js in the same folder as the PHP script and the HTML front end. Now that you’re familiar with jQuery, let’s move on to something more complicated: form elements and XML handling. This is true Ajax!

Example 2 - Chat with jQuery

To demonstrate the true power of jQuery, we’re going to make a fully featured Ajax chat application. This will allow users to post messages, and automatically update itself — all without any page refreshes. As we’re now dealing with a slightly more complex application, I’ll also take you deeper into jQuery, demonstrating other handy functions.

First, we’ll plan out the application. We won’t need much for this app — a front end, a back end, and the jQuery library. However, there will be a fair bit of code for both the back and front ends of the app, so our basic plan will let you know what to expect from the system.

Planning the Server Side

The back end of our application needs to process message submissions and output the messages. With this in mind, let’s put together a rough outline of the back-end code:

  • connect to database
  • if a message was submitted
    • insert message into database
    • delete old messages
  • fetch messages from the database and display as XML

As you can see, it’s all very simple and straightforward. You can use this plan as a guide if you have to write a back end in another language.

Planning the Client Side

The front end has to call the back end using Ajax, similarly to the approach we used in the first example. It has to handle the submission of the message-posting form, and update the chat window with newly submitted messages at regular intervals. However, we’re going to add another feature here — we’ll use the current UNIX timestamp to determine which messages have already been downloaded, and only fetch the new messages, reducing bandwidth usage and server load. Here’s a rough outline of the front-end code:

  • on page load
    • set current timestamp to 0 (all messages will be posted after this, i.e. all messages will be fetched)
    • call function to fetch new messages
  • function: fetch new messages
    • send request to server using POST
    • call function to parse XML response
    • add a timer to call this function after a second (increase frequency if server load is too high)
  • function: parse XML of new messages
    • set current timestamp as specified in the XML
    • if status code returned is ‘2′, no new messages, so end the function call
    • else, for each message in the response, add this to the top of the message window in this format:
      • author: message
  • on form submit:
    • send request to server using POST, specifying:
      • author name (user-specified)
      • message text (user-specified)
      • action noting that this is a post request
      • timestamp of the last request to the server
    • empty content of the message input box so that the user can start typing another message
    • call function to parse XML response (so that message posted is visible immediately)

This plan may seem far more complex than the back end, but thanks to jQuery, the code involved is fairly short.

Planning the Database

We’ll use a MySQL database to store the messages (although any SQL database will work, with a few tweaks to the code). We need a table with four columns: a column for the id of each message, two text columns to store the author of the message and the message itself, and a numerical timestamp column for our UNIX timestamps. Here’s the query that creates the table:

CREATE TABLE `messages` (
`id` int(7) NOT NULL auto_increment,
`user` varchar(255) NOT NULL,
`msg` text NOT NULL,
`time` int(9) NOT NULL,
PRIMARY KEY  (`id`)
);

Because we can’t tell what length the message will be, we’ll use a text field for now.

Server-side Code (XML)

In building the back end, let’s first decide what the back end should output (to determine the interface between the back end and front end), and work backwards from there. Here’s a simple XML structure:

<?xml version="1.0"?>
<response>
<status>1</status>
<time>1170323512</time>
<message>
<author>John Citizen</author>
<text>Hello world!</text>
</message>
<message>
<author>John Citizen</author>
<text>Hello world again!</text>
</message>
</response>

Notice that I’ve added the tag 'status', with the value of '1'. As I mentioned above, a status code of 1 will represent a successful request with new messages, 2 as successful without new messages. Each instance of the message tag includes the author and his or her message.

Server-side Code (PHP)

Now, to the back end. I’ll have to do this in PHP, but because the output is XML you can write the back end in any language — Perl, ASP, whatever you like. Let’s start it logically by defining some configuration values so that we can easily change them later. We need database connection details, the number of messages we want to store in the database (databases can handle thousands of rows, so this figure can be set fairly high), and the number of messages to display when the user enters the chat. Here’s the code:

$dbhost = "localhost“;
$dbuser = “root”;
$dbpass = “”;
$dbname = “chat”;
$store_num = 10;
$display_num = 10;

Now we need to get on to the basics of the back end itself. A database connection is required, but we also need to make sure that Internet Explorer doesn’t cache the request, and that the output is treated as XML. To make sure we’re able to identify any errors in the code, we’ll set error reporting to “all errors”. And to easily work with the request data, we’ll set a variable for every parameter in the request; each variable will take as its value the value of the request parameter. These few lines do the trick:

error_reporting(E_ALL);
header("Content-type: text/xml");
header("Cache-Control: no-cache");
$dbconn = mysql_connect($dbhost,$dbuser,$dbpass);
mysql_select_db($dbname,$dbconn);
foreach($_POST as $key => $value)
$$key = mysql_real_escape_string($value, $dbconn);

The foreach line looks through all the POST data, and creates a variable for every parameter and assigns it a corresponding value (e.g. path/to/file.php?variable=value would set $variable to "value"). This simplifies the process of grabbing request data, as we don’t have to specify it manually.

Next we get to the main functionality. It’s at this point that we handle the insertion of messages into the database, and the retrieval of the latest few messages based on the number of messages to be displayed, as defined in $display_num. I mentioned when we planned the front end that we would specify an action stating that a message was being submitted. We now need to check for this action — let’s assign the parameter 'action' a value of 'postmsg' to specify that we’re performing this check and insert the data as a new row in the database; we’ll insert the current UNIX timestamp in the database while we’re at it.

However, we also need to clean out the database. Depending on your database space limits, you may want to restrict the number of posts stored. Generally, logging of messages is frowned upon, so I’ve decided to store ten messages by default. We’ll use a function to grab the id of the last inserted row, and determine the rows to delete based on the value of that id. For example, if we insert the eleventh message, we’ll subtract the number of stored messages (10) from the id of the latest one (11) which gives us the id threshold (in this case, 1). We can then delete all messages that have an id equal to or less than that threshold, which in this example would result in us deleting the first message. And, thanks to SQL, we can do this all in one query.

Here’s the snippet that checks for the 'postmsg' action, inserts the message into the database, and cleans it out on the fly as well:

if(@$action == "postmsg")
{
mysql_query("INSERT INTO messages (`user`,`msg`,`time`)
VALUES ('$name','$message',".time().")");
mysql_query("DELETE FROM messages WHERE id <= ".
(mysql_insert_id($dbconn)-$store_num),$dbconn);
}

Developers using other server side technologies should be able to write equivalent code easily enough. Notice that we call the time function to grab the current UNIX timestamp. We can safely assume that the value that time returns will probably not change during script execution (even on a slow server, this script executes in under one hundredth of a second). So when we return a timestamp to the front end later, we can just call the time function again and the value should still be reliable.

The code that’s left handles the job of fetching the latest messages from the database and outputting them as XML. This is where the XML I outlined above comes into play. However, the bulk of the code lies in the MySQL query. We utilize the power of SQL to handle most of the processing tasks, so that script execution time isn’t affected. Here are the requirements for our SQL query:

  • It should fetch only the author and text of each message.
  • It should only fetch messages that have not been downloaded before — the client has a timestamp of the latest request, so this timestamp can be inserted into the SQL query.
  • It should order the messages so that the latest comes last, allowing them to be output in reverse order.
  • It should limit the number of messages fetched to the number defined in the configuration.

Anyone who’s familiar with SQL will agree that this is all fairly simple stuff. For the rest of us, here’s the code that does the trick. First, the query:

$messages = mysql_query("SELECT user,msg
FROM messages
WHERE time>$time
ORDER BY id ASC
LIMIT $display_num",$dbconn);

The rest of the code is elementary. If no results are returned, set the status code to 0; otherwise, set it to 1. Output the initial XML, the XML for each message, and the final XML. That’s all! Here’s the code:

if(mysql_num_rows($messages) == 0) $status_code = 2;
else $status_code = 1;

echo "<?xml version=\"1.0\"?>\n";
echo "<response>\n";
echo "\t<status>$status_code</status>\n";
echo "\t<time>".time()."</time>\n";
while($message = mysql_fetch_array($messages))
{
echo "\t<message>\n";
echo "\t\t<author>$message[user]</author>\n";
echo "\t\t<text>$message[msg]</text>\n";
echo "\t</message>\n";
}
echo "</response>";

The final code is all in the attached zip file, so don’t worry about copying this into your text editor. Now that the back end is finished, we can move on to the fun work — the HTML and jQuery!

Client-side Code (HTML)

Before we head into the jQuery, we need to prototype the HTML of the page. This way, when we have to work out which elements we are selecting to fetch or update using jQuery, we know what to do. We won’t need much: a wrapper div, a paragraph for the messages, and a form with fields for the user’s name and message, along with a Submit button. A briefly displayed loading message would add a final touch — we can remove this at the appropriate time using jQuery. Here’s the HTML:

<div id="wrapper">
<p id="messagewindow"><span id="loading">Loading...</span></p>
<form id="chatform">
Name: <input type="text" id="author" />
Message: <input type="text" id="msg" />
<input type="submit" value="ok" /><br />
</form>
</div>

Client-side Code (jQuery)

Now, on to the jQuery front end. First, we need to declare the timestamp of the current message as 0, and call the function that loads the messages from the server:

timestamp = 0;
updateMsg();

Next, we’ll write the code for the form submission. jQuery allows us to add an event hook for the form’s submit event, as though we were adding an onSubmit event within the HTML itself, except that we don’t have to touch the HTML. Here’s the submit event:

$("form#chatform").submit(function(){ /* Code */ });

Here we’re using CSS selector syntax to refer to the form element with an id of 'chatform'. As soon as we’re into the form submission code, we can fire off a POST request to the server using jQuery’s $.post. Within the call to $.post, we can select the values of the form elements on the page on the basis of their ids, as we determined earlier. With this in mind, we can dive into our Ajax call:

$.post("backend.php“,{ message: $(”#msg”).val(),
name: $(”#author”).val(), action: “postmsg”, time: timestamp }, function(xml) {

Notice that the array of parameters to be passed in the request is enclosed with curly braces. If you have more parameters, simply separate them with commas and use the format presented here, JSON-style. You can also use jQuery’s Ajax functions to send a GET request that expects a JSON-style response, and have jQuery convert the response text into an easy-to-use format. Bear in mind, though, that this capability is only available for the GET request type, not the POST request type we use for our chat system. As such, we’ll stick with plain XML for the moment.

Now, let’s look at handling the XML response. Since we’re all for code reuse, we’ll create a function that handles the XML and call it now:

addMessages(xml);

We’ll write this function later so that we can finish off the form submission event code. The code we’ve written so far is all we need for our $.post callback function, so we can close it off and add a return false; line. This line fails using the standard browser form submission error code. The browser doesn’t send the user to another page to submit the form — we’ve already handled the form submission, so the browser doesn’t need to. Here’s the event code in its entirety:

$("form#chatform").submit(function(){
$.post("backend.php",{
message: $("#msg").val(),
name: $("#author").val(),
action: "postmsg",
time: timestamp
}, function(xml) {
addMessages(xml);
});
return false;
});

Now, let’s get back to the addMessages() function, which handles the response xml. It’s quite simple, making use of jQuery’s DOM manipulation and traversing functions. Remember that I mentioned the status code before? Now’s the time to handle it:

if($("status",xml).text() == "2") return;

I haven’t mentioned context in jQuery yet. The XML in this function call tells jQuery to look not inside the document HTML, but in the XML that the server sent us.

That line of code checks for a status code of 2, representing a successful request that resulted in us having no new messages to add to the window. The 'return' keyword terminates the function call. Next, we set the timestamp to the timestamp in the XML:

timestamp = $("time",xml).text();

Again, this fetches the text value of the <time> tag in the XML.

Now we can move on to jQuery’s array iteration function, each(). jQuery has an interesting way of handling array iteration. We use a standard selector statement, and the each() function is passed one parameter — a function to handle each instance of the matched elements. In this case, the elements are the instances of the <message> tag in the server response, each instance representing a message to be displayed. One parameter — the id of the instance — is passed to the function. We can use this, with jQuery’s get() function, to grab a new context — the actual XML of the <message> tag. This is how we select it:

$("message",xml).each(function(id) {
message = $("message",xml).get(id);

We can then select elements by passing the context 'message' to the jQuery / $ function. Now that we have all the data we need, we must add it to the top of the message window on the page. The message window has the id 'messagewindow', so we select it using $("#messagewindow") and use the prepend() function to add our data:

$("#messagewindow").prepend("<b>"+$("author",message).text()+
"</b>: "+$("text",message).text()+
"<br />");

That’s all there is to it! Putting it all together, here’s the code for the function:

function addMessages(xml) {
if($("status",xml).text() == "2") return;
timestamp = $("time",xml).text();
$("message",xml).each(function(id) {
message = $("message",xml).get(id);
$("#messagewindow").prepend("<b>"+$("author",message).text()+
"</b>: "+$("text",message).text()+
"<br />");
});
}

Finally, we need the updateMsg function we called at the very beginning of our code. This function has to query the server for new messages, and call the above addMessages function with the response. It also has to set a timeout to call itself after a set period of time, which makes the chat window update automatically. To begin, we don’t need to submit anything to the server besides a timestamp, so this is our $.post call:

$.post("backend.php",{ time: timestamp }, function(xml) {

As I noted before, we also need to remove the loading message at this point, so we call jQuery’s remove function on the span:

$("#loading").remove();

Then, we’ve received our xml response in the object 'xml', so we pass it to our addMessages function:

addMessages(xml);

We round it off with a call to the JavaScript setTimeout() function, which executes specified code after a specified interval. Here’s the whole function put together:

function updateMsg() {
$.post("backend.php",{ time: timestamp }, function(xml) {
$("#loading").remove();
addMessages(xml);
});
setTimeout('updateMsg()', 4000);
}

Putting it All Together

Now we can put all of the pieces of the puzzle together. The code is, as I mentioned, available in this downloadable zip file. However, you can peruse it here, where I’ve added a bit of HTML and CSS for layout:

<html>
<head>
<title>Ajax with jQuery Example</title>
<script type="text/JavaScript" src="jquery.js"></script>
<script type="text/JavaScript">
$(document).ready(function(){
timestamp = 0;
updateMsg();
$("form#chatform").submit(function(){
$.post("backend.php",{
message: $("#msg").val(),
name: $("#author").val(),
action: "postmsg",
time: timestamp
}, function(xml) {
$("#msg").empty();
addMessages(xml);
});
return false;
});
});
function addMessages(xml) {
if($("status",xml).text() == "2") return;
timestamp = $("time",xml).text();
$("message",xml).each(function(id) {
message = $("message",xml).get(id);
$("#messagewindow").prepend("<b>"+$("author",message).text()+
"</b>: "+$("text",message).text()+
"<br />");
});
}
function updateMsg() {
$.post("backend.php",{ time: timestamp }, function(xml) {
$("#loading").remove();
addMessages(xml);
});
setTimeout('updateMsg()', 4000);
}
</script>
<style type="text/css">
#messagewindow {
height: 250px;
border: 1px solid;
padding: 5px;
overflow: auto;
}
#wrapper {
margin: auto;
width: 438px;
}
</style>
</head>
<body>
<div id="wrapper">
<p id="messagewindow"><span id="loading">Loading...</span></p>
<form id="chatform">
Name: <input type="text" id="author" />
Message: <input type="text" id="msg" />
<input type="submit" value="ok" /><br />
</form>
</div>
</body>
</html>

So, with 22 lines of JavaScript, eight lines of HTML, and around 50 lines of PHP, we now have a fully functional Ajax web application. Try it out, and integrate it into your own site. Build your own Ajax application, using these techniques and your own ideas. Take this code and modify it to build something new. If you aren’t comfortable with generating and handling XML, stick with using your web application to generate HTML, and using load() to bring it to the client. Then, once you get the hang of it, try out an application that utilises the full power of XML using tag attributes and jQuery’s attr() function — you’ll be amazed at just how powerful Ajax with jQuery can be.

Whip Up a Yahoo! Mashup Using PHP

Posted in Software on September 4th, 2007 by Valeriu Buzila / No Comments »

So, you want to create your own mashup? Great! Mashups — web applications that grab information from different external sources and mix it together in new and exciting ways — are fun to build, popular with users, and oh-so-very Web 2.0.

Web services, a collection of standards and data formats that enable web applications to communicate with each other and share data, are the core technology behind mashups. They allow developers to directly feed data from databases without having to resort to messy screen-scraping, and enable developers to build rich content applications that make information more useful to users.

These days, numerous companies publish public APIs for their various web services. In this article, I’ll show you how to use the powerful collection of Yahoo! APIs to build a mashup with PHP 5. First we’ll take a look at what APIs are, and the various offerings from Yahoo! that we can take advantage of. I’ll demonstrate how to search the web using Yahoo!’s entire database with only three lines of code, then take you through the process of building an entire application to search for ‘Pizza’ in ‘Palo Alto, CA’ with only 25 lines of PHP code.

This tutorial assumes a decent knowledge of PHP 5 — specifically, that you’re comfortable with the basic syntax, classes and objects, and have some basic background on web services. Take a look at the Wikipedia article on web services and SitePoint’s other PHP tutorials if you think you may have trouble with these.

The Yahoo! Developer Network

The Yahoo! Developer Network is the central reference site for all Yahoo! APIs and a way of connecting with developers and enabling them to provide Yahoo!-powered services in their applications. Check it out for more in-depth reference information.

Available APIs

A quick glance through the sidebar of the developer network homepage reveals just how many APIs Yahoo! provides. Many of the standard Yahoo! services have public APIs available, such as Search, Maps and Answers, and each API has its own set of documentation. A number of startups acquired by Yahoo! also provide APIs to their services, including del.icio.us, Flickr and Upcoming.org, although each of these APIs is handled differently — check their documentation for details. Here’s a quick summary of the most popular ones.

Flickr

One of the most popular APIs available from Yahoo!, the Flickr web services offer access to all the photos on Flickr, photo meta data, tags and more. Popular uses of the API include visualizing locations on maps, galleries, posters and games. You can find links to examples on the Flickr services page including Retrievr, an image search engine that allows you to search for images matching your hand-drawn sketch.

Searching for images on retrievr.com (click to view image)

Yahoo! Maps

The Yahoo! maps API is extremely powerful and allows you to create interactive maps with high quality geographic imagery, adding the power of visual location to your web application. Popular applications of the API include mapping journeys and locating users. Runningmap.com is an excellent example of the Yahoo! Maps API, allowing you to plot your exercise routes and check your distances and other interesting statistics.

A map on Runningmap.com (click to view image)

Yahoo! Search

Want to build a search function for your web site, or add related links to your web application? One of the more traditional APIs from Yahoo!, the Search API allows powerful querying of the Yahoo! web search database. Christian Langreiter offers a comparison of Yahoo and Google search results in one of his mashups.

The application compares Yahoo! and Google results (click to view image)

Each API has different usage restrictions, so check the relevant guidelines before you begin to develop an application. Some APIs have rate limiting — for example, the Web Search API is restricted to 5000 queries per IP address per 24 hour period. In addition, not all APIs can be used for commercial purposes. Yahoo! requires developers to register their applications with the developer network to receive an application ID, and include this application ID in each request they make to an API. While this data doesn’t affect rate limiting, it allows Yahoo! to monitor API usage and contact developers if needed. You can get an application ID — and you’ll need one to try out the sample code in this article.

Consuming the Yahoo! APIs

All the standard Yahoo! APIs are RESTful in nature; data is accessed via standard HTTP requests, where query parameters are passed to the API through the URL. What this means is that you can fetch data from the API as easily as you would fetch the HTML source of any web page, and you can test requests using any browser that will render XML. Take this sample URL:

http://api.search.yahoo.com/WebSearchService/V1/webSearch?
appid=YahooDemo&query=SitePoint&results=2

Try it out — visit the URL in your browser and take a look at the XML output. Since the output is XML, however, we need to parse the response XML with PHP in order to make use of it. A number of classes and XML parsing functions are readily available for PHP, but Yahoo! goes a step further: it provides an option to get all the output data in serialized form. This means that you can quickly fetch the data, run it through the unserialize function and start working with the data immediately. To get PHP serialized output, just append &output=php to the query string. Here’s a simple script that searches the web for “SitePoint” using the web search API, all in three effective lines of code:

<?php
$output = file_get_contents(
'http://api.search.yahoo.com/WebSearchService/V1/'.
'webSearch?appid=your-app-id-here'.
'&query=SitePoint'.
'&results=2'.
'&output=php'
);
$output = unserialize($output);
echo '<code>'.print_r($output,TRUE).'</code>';
?>

Examining this snippet, we see that we first fetch the data in PHP format using the file_get_contents function, then convert it to an associative array with unserialize, before outputting a dump of the array with print_r. I should note here that to be able to use file_get_contents to open a URL requires the allow_url_fopen setting to be enabled in php.ini. Check with your host if you’re not sure if this setting is in place. Let’s take a look at the output. $output['ResultSet']['Result'] contains the data we’re looking for. Here’s a snippet of the first search result:

[Title] => SitePoint : New Articles, Fresh Thinking for Web Developers and Designers
[Summary] => Network of sites that provice information, tools, and resources for internet-focused businesses and web developers, including WebmasterBase.com, eCommerceBase.com, and PromotionBase.com.
[Url] => http://www.sitepoint.com/

Compare this with the first result for the term “SitePoint” on the official Yahoo! Search site and you’ll see it’s the same.

Result for a search on (click to view image)

All the data you would normally get through a Yahoo! search is available through the API! Forget messy screen scraping — Yahoo! makes it easy to get the search results directly. With another few lines of code, we can turn our array into a full search results page. You can probably already see how to make a search engine with the Yahoo! web search API here — it’s that simple! Building your own search engine from scratch would usually involve buying a data centre and spending years spidering the web, but Yahoo! provides all the data free of charge (with usage restrictions, of course). You may have heard of rollyo.com, a site that lets you create your search engine. Well, Rollyo is built using the Yahoo! APIs. With just a few lines of PHP, we can query the Yahoo! web search API, parse the API output, extract the data, format it and output it — that’s a lot of power to have available at your fingertips.

Most of the Yahoo! APIs are just as easy to use as Web Search, and Yahoo! provides a guide to constructing REST queries if you need further information about fetching data from the APIs. Each API method has a documentation page outlining the parameters available, return values, possible output formats (XML, PHP, etc.) and potential errors. It should be noted that all parameters are URL encoded, so php code should become php+code, or the API may return unexpected output. PHP’s inbuilt urlencode function is sufficient for this task. As a general guide, each service has a base URL something like the following:

http://api.search.yahoo.com/WebSearchService/V1/webSearch

Each service requires certain parameters, one of which will be your application ID, and most services will have a query parameter. In the previous example search, we use the parameters appid, query, results and output. In this case, appid is the application ID, query is what we are searching for (”SitePoint”) and output is the format we want the API output in (php for serialized PHP form, optional). The results parameter is optional, however I use it here to limit the output to two search results in order to reduce server load. If you don’t need the standard 10 results, limiting the output through the results parameter is highly advisable. We append a question mark and the parameters to the URI, separating each parameter with an ampersand (&) and using the standard option=value format, just like any other HTTP request.

So now that you’ve had a gentle introduction to the usage of Yahoo! APIs, how they behave and what they provide access to, let’s take things up a notch and look at how we can actually put these APIs to good use.

PHP 5 and the Yahoo! APIs

PHP 5 has a number of features that help us make effective use of the Yahoo! APIs. PHP 5’s improved internal OOP support enables developers to efficiently build applications, and we can take advantage of this by using classes to rapidly develop libraries that make use of the Yahoo! APIs. The introduction of the file_get_contents function allows easy querying of the APIs, and as I mentioned before, PHP has the added advantage of receiving serialized output from most of the APIs with inbuilt parsing functions. Even though the HTTP extension is available in PHP 5, we won’t need it for low-end API consumption. PHP 5 developers can easily query the APIs, parse the output, deal with errors and make use of the data without too much trouble.

Quick and Easy Mashups

To demonstrate the power of the Yahoo! APIs, we’re going to put together a very simple, practical application using PHP 5 and various Yahoo! APIs. What we want to do is query the local search API for data from Yahoo! Local, where users discuss and rate local attractions, businesses and so on. Then we’ll place that data on a map, showing geographically where those attractions are located.

YahooAPI Client Class in PHP 5

To make our lives easier, we’re going to use a PHP class that helps us to query the Yahoo! APIs. The base class needs to have the following functionality:

  • Set the web service to be used.
  • Add parameters to the request.
  • Execute the call to the remote API.
  • Fetch the output.

I’ve built a very simple, extensible class to do just this, called YahooAPI — take a look in the code archive for this article. While I won’t go into the details of the YahooAPI class, using it is very easy. In the previous example we searched for “SitePoint” on the Web. The same task can be achieved very easily with the class:

$api = new YahooAPI();
$api->setAppID('your-app-id-here');
$api->setService('http://api.search.yahoo.com/WebSearch
Service/V1/'.
'webSearch');
$api->setParam('output','php');
$api->setParam('query','SitePoint');
$api->setParam('results','2');
$output = $api->doAPICall();

This is exactly the same as the previous search example, except that now we use the client class. In this case, using the class is more verbose, but with more complex and repeated calls to the API, using the class can save time and simplify maintenance.

Manipulating Data from API Calls

Now that we have everything we need to get to work, I’ll show you how to easily manipulate data from the Yahoo! APIs.

Let’s use our new YahooAPI class to search for ‘Pizza’ in ‘Palo Alto, CA’ using the Local Search API. Take a look at the documentation page for the latest version of the local search API. This is the base URL for the service:

http://local.yahooapis.com/Local
SearchService/V3/localSearch

So we’ll call the setService method of the YahooAPI class and give it the base URL. Looking through the request parameters in the documentation, we’ll need to submit the 'appid', 'query' and 'location' parameters. After we set the required application ID, we then need to choose a location — we’ll use a city and state for now — and a query. Let’s say we’re searching for ‘Pizza’ in ‘Palo Alto, CA’. In simple, procedural PHP code using the YahooAPI class, the search would look something like this:

$api = new YahooAPI();
$api->setService('http://local.yahooapis.com/LocalSearch
Service/V3/'.
'localSearch');
$api->setAppID('your-app-id-here');
$api->setParam('output','php');
$api->setParam('query','pizza');
$api->setParam('location','Palo Alto, CA');
$output = $api->doAPICall();

The call to the API here is the same as fetching

http://local.yahooapis.com/LocalSearch
Service/V3/localSearch?appid=your-app-id-here&
query=pizza&location=Palo+Alto,+CA

through any method, so take a look at that URL in your web browser. Clearly the information we’re looking for is within each <Result> node, and all <Result> nodes are within one big <ResultSet> node. The PHP version that we’re fetching has exactly the same structure as the XML, except that it’s in an easily usable array. After fetching all that data into the $output variable, all we have to do is iterate over the $output['ResultSet']['Result'] elements and fetch the data we need. Try it out — add the following code after the previous example and run it on your web server:

foreach($output['ResultSet']['Result'] as $result) {
echo $result['Title'].'<br/>';
}

You should receive something like the following output:

Patxi's Chicago Pizza
Papa Murphys Pizza Take & Bake
New York Pizza
Round Table Pizza Palo Alto
Domino's Pizza
California Pizza Kitchen
Pizza My Heart
Ramonas Pizza
Round Table Pizza Palo Alto
Spot A Pizza

LocalSearch Client Class in PHP 5

Now let’s extend the YahooAPI class to query the Yahoo! Local Search API. We can easily create a class that manages all this work for us. Instead of the previous procedural code, if we extend the YahooAPI class and create some appropriately-named methods — for example locationSearch($query, $location) — we can further simplify the process of interacting with the APIs.

I’ve written an example class that generally covers everything we need for interacting with the Local Search API. It has three main methods: locationSearch, positionSearch and extractResults. Here’s the code:

<?php
require_once('yahooapi.class.php');
class LocalSearch extends YahooAPI
{

The constructor method sets the basic properties we’ll always need for each Local Search API request:

 public function __construct()
{
$this->setParam('output','php');
$this->setService('http://local.yahooapis.com/'.
'LocalSearchService/V3/localSearch');
$this->setAppID('your-app-id-here');
}

The two search methods represent different ways of performing a search, depending on whether or not we have a location name or exact GPS coordinates. They simply set the required parameters and call the doAPICall method of the parent YahooAPI class:

 public function locationSearch($query,$in_location)
{
$this->setParam('query',$query);
$this->setParam('location',$in_location);
return $this->doAPICall();
}

public function positionSearch($lat,$long)
{
$this->setParam('query','*');
$this->setParam('latitude',$lat);
$this->setParam('longitude',$long);
return $this->doAPICall();
}

The extractResults method saves us entering ['ResultSet']['Result'] all the time when we want to output the results, because the data we need will always be within that part of the returned array:

 public function extractResults()
{
$return = $this->getResults();
$return = $return['ResultSet']['Result'];
return $return;
}
}
?>

You’ll find a copy of the code in the archive for the article. Keep it handy because we’ll be using it in our example mashup. We’ll also build a similar class for the Maps API later, as it has some slightly different requirements.

Our example “pizza” search, executed using our new LocalSearch class, now looks like this:

require_once('localsearch.class.php');

$localSearch = new LocalSearch();
$localSearch->locationSearch('Pizza','Palo Alto, CA');
$output = $localSearch->extractResults();

That’s much easier, don’t you think?

Yahoo! Maps AJAX API

Perform a quick web search and you could find the web sites for these pizza places, each of which would list the restaurant’s address. But why would you go to all that trouble when Yahoo! provides all this data in the returned array — as well as latitude and longitude information? Here’s an example of the data returned:

[Title] => Patxi's Chicago Pizza
[Address] => 441 Emerson St
[City] => Palo Alto
[State] => CA
[Phone] => (650) 473-9999
[Latitude] => 37.445265
[Longitude] => -122.163432

Now that we have the location information, we need to figure out how to plot it on a map. Yahoo! provides a map image API, offering raw images of maps in PNG format, and (in theory) we could use GD to draw markers on the map. However, since we’re building a web application, we can instead use the Maps AJAX API to generate a UI-friendly, interactive Yahoo! maps display. We can then add markers to the map in preset positions, using the bundled functions. (Note that the Maps AJAX API isn’t really an Ajax API! In fact, there isn’t any real Ajax at all — that is, there are no XMLHttpRequest calls — but due to the fact that the term Ajax has come to represent any web page technology that uses JavaScript and doesn’t need to reload the web page to update itself, Ajax would be the best way to describe it.)

Unlike the other Yahoo! APIs, the Maps AJAX API isn’t a REST-based web service. Instead, you have to include a JavaScript file on your page, create an instance of the map in a container on the page (usually a <div>) and manipulate it through JavaScript calls. Luckily, the HTML and JavaScript required are quite simple. Take a look at the mapoutput.php example in the code archive. This is all the JavaScript you’ll need to generate the map:

var ymap = new YMap(document.getElementById('mC'),YAHOO_MAP_REG);
var mPoint = new YGeoPoint(37.4041960114344,-122.008194923401);
ymap.drawZoomAndCenter(mPoint, 3);
var marker = new YMarker(mPoint);
marker.addLabel('A');
marker.addAutoExpand('<div class="mp">Some Text</div>');
ymap.addOverlay(marker);

The first line creates a new instance of the YMap class, and assigns it to the element on the page with an ID of 'mC'. The second argument represents the desired map type, in this case a regular map rather than satellite imagery, YAHOO_MAP_SAT, or a hybrid of the two, YAHOO_MAP_HYB. The second line creates a YGeoPoint object, a point on the map based on latitude and longitude coordinates, while the third line calls the map object and tells it to centre itself on this new point and display the map, at a zoom level of 3.

The final three lines of JavaScript create a YMarker object, a visual map marker that expands to reveal some extra content when it’s moused over. Our challenge is to generate all this with PHP, and to make the job easy, we’re going to build a class that generates all the code for us.

Mashup Time!

Now that we’ve sorted out how to query the APIs and deal with the data, it’s mashup time! As I mentioned before, we’ll create a simple application that finds places with the local search API and, using the latitude and longitude coordinates from the returned search data, marks the exact locations of these places on a map from the maps API. And of course we’ll wrap all the functionality up in a simple PHP class. The AJAX map API uses JavaScript code, so we’ll use our PHP class to generate the required JavaScript based on the returned search data.

First, we’ll start by querying the Maps API. I’ll use the client class for the local search API, which I demonstrated earlier, to execute this sample query: ‘Pizza’ in ‘Palo Alto, CA’. We need an instance of the LocalSearch class, and we’ll use its locationSearch method to execute the API query. The extractResults method will give us the data that we want to work with. Now that we’ve built our client classes, all this can be achieved in three lines of code:

$localSearch = new LocalSearch();
$localSearch->locationSearch('Pizza','Palo Alto, CA');
$apiOutput = $localSearch->extractResults();

Let’s take a step back for a moment and call print_r($apiOutput) to see what we have. The array $apiOutput now contains a number of sub-arrays, each of which is a single search result and contains (among other things) a ‘Title’, ‘Latitude’ and ‘Longitude’. That’s all we need for the moment, so let’s quickly extract this information and delete the rest:

foreach($apiOutput as $id => $result)
{
$points[$id] = array($result['Title'],
$result['Latitude'],
$result['Longitude']);
}

Here are some sample values from our newly defined $points array:

[0] => Array
(
[0] => Patxi's Chicago Pizza
[1] => 37.445265
[2] => -122.163432
)

[1] => Array
(
[0] => Papa Murphys Pizza Take & Bake
[1] => 37.433243
[2] => -122.129291
)

For each marker we want to put on our map, we need to know its position, and we need some summary text that we can have appear when the user mouses over it. In this case, when visitors mouse over one of our markers, we’ll show them the name of the place.

Now that we have all our locality information, we come to the tricky bit — generating the map HTML and JavaScript. Basically, our map code consists of two distinct sections — the <head> HTML and CSS (for the map container), and the <body> HTML and JavaScript. The <body> includes the container <div> for the map and some JavaScript for the Maps AJAX API. For each marker we want to add to the map, we need a JavaScript YGeoPoint object to define its position, and a YMarker object to be the marker itself. We then customise the marker through the addLabel and addAutoExpand methods (many more are documented here) before placing it on the map using the map object’s addOverlay method. We’ll now create a PHP class that takes care of generating all of this JavaScript code, and call it AjaxMap.

Here’s a summary of the methods our class will have:

  • getHeadHTML for generating <head> code
  • getMapScript for generating <body> code
  • initMarker for generating code for each marker

We’ll also add some helper methods for customising the map:

  • setMapType for choosing between maps, satellite images, and hybrids
  • setMapContainer for setting the ID of the map container <div>
  • addMarker for adding markers to the map

Let’s begin our AjaxMap class:

<?php
require_once(’yahooapi.class.php’);
class AjaxMap
{
private $mapContainer;
private $mapType;
private $markers = array();
public $showZoom;
public $showPan;

We begin by including the YahooAPI class, which we’ll use to generate the Yahoo! API calls. Our class has three private properties: $mapContainer will contain the ID of the HTML element acting as the map container, $mapType will represent the type of map desired and must be one of YAHOO_MAP_REG, YAHOO_MAP_SAT or YAHOO_MAP_HYB, and the final private property, $markers will contain an array of map location markers. The API offers the ability to add zoom and pan controls, so we’ll add the public properties $showZoom and $showPan, which can be set to true when required.

So, first to the easy methods: getHeadHTML, the set functions and addMarker. All the getHeadHTML method needs to do is return a <script> tag referencing the Yahoo! AJAX Map API:

public function getHeadHTML()
{
return '<script type="text/javascript” ‘.
’src=”http/api.maps.yahoo.com/ajaxymap?v=3.0&appid=your-app
-id-here”>’.
“</script>\n”;
}

The set functions are just as simple — they act as wrapper methods for modifying private properties. Here’s the code:

 public function setMapContainer($id)
{
$this->mapContainer = $id;
}
public function setMapType($type)
{
$this->mapType = $type;
}

The addMarker method will add a new map marker entry to the private $markers array and takes a latitude value, a longitude value and description text as its arguments:

 public function addMarker($lat,$long,$descr)
{
$this->markers[] = array($lat,$long,$descr);
}

initMarker is a private method called for each of the desired map markers and generates the JavaScript code required for the marker:

 private function initMarker($id,$lat,$long,$descr,$init_geo = TRUE)
{
$js = '';
if($init_geo) $js .= "\nvar mPoint$id = ".
"new YGeoPoint($lat,$long);\n";
$js .= "var currmarker = new YMarker(mPoint$id);\n";
$js .= "currmarker.addLabel('$id');\n";
$js .= "currmarker.addAutoExpand('<div class=\"mp\">".
addslashes($descr)."</div>');\n";
$js .= "ymap.addOverlay(currmarker);\n\n";
return $js;
}

initMarker takes all the information about the marker — latitude and longitude for position, a short description and some notes, plus a unique 'id' parameter — and generates the JavaScript we need in order to draw the marker. The $init_geo parameter for initMarker indicates whether or not we need to create a YGeoPoint object for the marker; this may already have been done.

All that’s left to do is bring everything together within the main JavaScript block. The getMapScript method will generate this JavaScript and assign it to the $js variable:

 public function getMapScript()
{
$js = '';

First, we have to initialise a YMap object. This is our main map object which will handle the drawing and customisation of the map. The first part is simple — we output the code required to create a new YMap object:

   $js .= 'var ymap = new YMap(document.getElementById(\''.
$this->mapContainer.'\'),'.$this->mapType.");\n";

In this instance, the properties $mapContainer and $mapType include the relevant information about the map, so setMapType and setMapContainer should be called before getMapScript.

Next, we output the JavaScript to add the zoom and pan controls if $showZoom and $showPan are set to true. To add a zoom control in JavaScript, we use the addZoomShort method of the YMap object, and addPanControl for a pan control:

   if($this->showZoom) $js .= "ymap.addZoomShort();\n";
if($this->showPan) $js .= "ymap.addPanControl();\n";

We may have a number of markers to display, but the map can only be centred on one of them. To keep it simple, we’ll remove the last marker from the main set of markers, centre the map on it and draw it on the map before proceeding to draw the remaining markers. Obviously, none of this is needed if no markers are to be drawn on the map, so we check that markers exist here too. Here’s the code that outputs the JavaScript required for centring the map on the last marker, and drawing that marker:

   if(count($this->markers) > 0)
{
$lastmarker = array_pop($this->markers);
$js .= 'var mPoint'.count($this->markers).' = new YGeoPoint('.
$lastmarker[0].','.$lastmarker[1].");\n";

$js .= 'ymap.drawZoomAndCenter(mPoint'.count($this->markers).
", 3);\n";

$js .= $this->initMarker(count($this->markers), $lastmarker[0],
$lastmarker[1], $lastmarker[2],
FALSE);

First we check if there are more than 0 markers (that is, we see if any have been set), and if so, extract the last of these markers and use that marker’s data to write the JavaScript required to create a new YGeoPoint object. We then output the JavaScript required to draw the map, centre it on our last marker and set the zoom level to 3. In JavaScript, we do this via the drawZoomAndCenter method of the YMap object. We then call our initMarker function to generate the rest of the JavaScript, and through its last parameter, tell it not to output the JavaScript to create a YGeoPoint object, as we’ve already taken care of it.

Finally, we generate the code for each remaining marker by quickly iterating over the markers array, calling initMarker for each one and returning the $js string variable:

     foreach($this->markers as $id=>$obj)
{
$js .= $this->initMarker($id,$obj[0],$obj[1],$obj[2],TRUE);
}
}
return $js;
}
}
?>

That also represents the end of our AjaxMap class!

Now we just have to use our local search class and AjaxMap class in a proper application. I’ve put together a quick demonstration. First we need to include our two classes:

<?php
require_once('localsearch.class.php');
require_once('ajaxmap.class.php');

Next, we use our local search class to search for “pizza”. We collect the locations from our search results and store them in an array called $points:

$localSearch = new LocalSearch();
$localSearch->locationSearch('Pizza','Palo Alto, CA');
$apiOutput = $localSearch->extractResults();

foreach($apiOutput as $id => $result)
{
$points[$id] = array($result['Title'],
$result['Latitude'],
$result['Longitude']);
}
unset($apiOutput);

We then create a new AjaxMaps object and add to it all our locations:

$ajaxMap = new AjaxMap();
$ajaxMap->setMapContainer('mC');
$ajaxMap->setMapType('YAHOO_MAP_REG');
$ajaxMap->showPan = true;
$ajaxMap->showZoom = true;

foreach($points as $point)
{
$ajaxMap->addMarker($point[1],$point[2],$point[0]);
}
?>

Now that all our location markers have been added to our AjaxMap object, the only task that’s left to do is write the page HTML and output the JavaScript:

<!DOCTYPE html public "-//W3C//DTD XHTML 1.0 Transitional//EN”
“http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd”>
<html xmlns=”http://www.w3.org/1999/xhtml”>
<head>
<?php echo $ajaxMap->getHeadHTML(); ?>
<style>
#mC {
height: 500px;
width: 500px;
}
.mp {
width:160px;
height:50px;
}
</style>
</head>
<body>
<div id=”mC”></div>
<script type=”text/javascript”>
<?php echo $ajaxMap->getMapScript(); ?>
</script>
</body>
</html>

The entire script is available in the code archive. Set your application IDs, load it up on your web server, and with any luck you should see something like this:

Our app in action! (click to view image)

Congratulations! You’ve just built a mashup using the Yahoo! APIs. With a bit of tweaking, you could add search functionality, allowing the user to look for more than just ‘Pizza’ in ‘Palo Alto, CA’. You could even integrate the functionality into an existing application (although be aware of the terms of use for both APIs). The possibilities are endless.

Where to From Here?

As you can see, exploiting the Yahoo! APIs with PHP5 to create useful mashups is a piece of cake, and there are many interesting applications that can be built with the data. Rasmus Lerdorf himself has written a similar article, taking a more in depth look at the Yahoo! Geocoding API, and how to easily use it with PHP5. It’s also worth noting that while we’ve used the output=php parameter throughout this article, most of the APIs also offer JSON output for use via Ajax. The Yahoo! Developer Network’s PHP Developer Center has an excellent collection of tutorials, code samples and other resources for consuming the APIs with PHP5.

Also check out Yahoo’s Application Gallery for inspiration and to see some great web apps built with various Yahoo! APIs, If you build an interesting application, submit it to the gallery for some excellent exposure and useful feedback

Generating Spreadsheets with PHP and PEAR

Posted in Software on September 4th, 2007 by Valeriu Buzila / No Comments »

In the last article Getting Started with PEAR, you got the PEAR Package Manager up and running. Now, it’s time to put PEAR to good use with PEAR::Spreadsheet_Excel_Writer, a library for generating Excel spreadsheets.

Here’s what we’ll cover today:

  • Introducing PEAR::Spreadsheet_Excel_Writer
  • Finding your way around: introducing the API
  • Adding Cell Formatting: eye candy matters
  • Adding Excel Functions: C1 + D1 = 2!

Be aware that I’m assuming you have a rudimentary knowledge of Excel, but nothing too serious. You don’t even have to own a copy — all the examples here work equally well with OpenOffice Calc.

Introducing PEAR::Spreadsheet_Excel_Writer

Let’s face it — although you and I may be preoccupied with XML markup, tabbed browsing, editors like Emacs and VI, and minimizing the use of system resources, the rest of the computer-using world is happily chugging away with the likes of Microsoft Office. And although they may be moderately impressed by the amazing things you can do with an HTML table, when it comes to dealing with numbers, Excel is pretty much a standard.

More to the point, Excel is widely used by those dealing with finance and money. In other words, the Accounts department that’s failed to pay your bill on time is probably using it. Make the accountants’ lives easier, and they might return the favour…

Wouldn’t it be great if you could provide your customers access to downloadable numerical data in the form of an Excel spreadsheet? The good news is that you can, with PEAR::Spreadsheet_Excel_Writer.

“Impossible!” you cry. “Excel uses some proprietary Microsoft file format. This can’t be done!”

Yes, it can. Spreadsheet_Excel_Writer generates the “real thing”, complete with Excel functions, formatting and all. No, we’re not talking generating comma separated files here, or using the COM extension (or any other extension, for that matter). This is written in pure PHP and will work just as well from a UNIX-based Web server as it does from a Windows-based server — and there’s no need to bug your hosting provider. In short, PEAR::Spreadsheet_Excel_Writer, with additional magic from PEAR::OLE, “understands” Microsofts Excel file formats.

Let’s take a moment to tip our hats to Xavier Noguer, who’s done an amazing job in bringing all this to PHP, with help from Mika Tuupola for Spreadsheet_Excel_Writer.

Now, without further ado, and armed with full knowledge of PEAR’s package manager, which you installed successfully last month (right?), let’s start by downloading the libraries. Open up your command prompt and type:

$ pear install OLE
$ pear install Spreadsheet_Excel_Writer

That’s it. We’re ready for action!

Important Note! I used PEAR::OLE version 0.5 and PEAR::Spreadsheet_Excel_Writer version 0.7 for the examples in this article. Be warned that things may change with future releases.

Finding your Way Around

To kick things off, let’s generate a very simple spreadsheet.

<?php
// Include PEAR::Spreadsheet_Excel_Writer
require_once "Spreadsheet/Excel/Writer.php";

// Create an instance
$xls =& new Spreadsheet_Excel_Writer();

// Send HTTP headers to tell the browser what’s coming
$xls->send(”test.xls”);

// Add a worksheet to the file, returning an object to add data to
$sheet =& $xls->addWorksheet(’Binary Count’);

// Write some numbers
for ( $i=0;$i<11;$i++ ) {
// Use PHP’s decbin() function to convert integer to binary
$sheet->write($i,0,decbin($i));
}

// Finish the spreadsheet, dumping it to the browser
$xls->close();
?>

Filename: example_1.php

Point your browser to the script and, assuming it knows about Excel (or OpenOffice Calc), up pops a spreadsheet, containing the numbers 0 to 10 as binary.

Storing Files

The spreadsheet is dynamically rendered in this case — nothing is stored on the server. If you want to generate a file instead, you can cut out the processing required to generate a sheet that hasn’t changed simply by passing the constructor a legal path and filename, and avoiding sending the HTTP headers, like so:

<?php
// Has a spreadsheet been created?
if ( !file_exists('sheets/binary.xls') ) {

// Include PEAR::Spreadsheet_Excel_Writer
require_once “Spreadsheet/Excel/Writer.php”;

// Create an instance, passing the filename to create
$xls =& new Spreadsheet_Excel_Writer(’sheets/binary.xls’);

// Add a worksheet to the file, returning an object to add data to
$sheet =& $xls->addWorksheet(’Binary Count’);

// Write some numbers
for ( $i=0;$i<11;$i++ ) {
// Use PHP’s decbin() function to convert integer to binary
$sheet->write($i,0,decbin($i));
}

// Finish the spreadsheet, dumping it to the browser
$xls->close();
}
?>

Your spreadsheet is ready for download here

Filename: example_2.php

If you’re using a UNIX-based system, remember to modify the permissions of the directory in which you’re storing the spreadsheet, so PHP can write to it.

API Overview

OK, we’re done with the basics. To get the most out of PEAR::Spreadsheet_Excel_Writer, you need to know a little bit more about the API, though. The API documentation that is available on the PEAR Website is out of date right now (it’s grown a lot, it seems, since that version of the documentation was generated). Thankfully, the authors have, for the most part, added inline documentation to the code, so you can make your own API docs by downloading phpDocumentor and pointing it at a directory that contains all the
Spreadsheet_Excel_Writer source code. If you need help getting started with phpDocumentor (and will excuse the sales pitch), it’s discussed in Volume 2: Applications of The PHP Anthology.

The main class that you’ll always begin work with, Spreadsheet_Excel_Writer, represents the point of access to all other classes in the library. It provides two important factory methods (which are actually defined in the parent class Spreadsheet_Excel_Writer_Workbook:

  • addWorksheet() - returns an instance of Spreadsheet_Excel_Writer_Worksheet. A large part of the work is done with instances of this class (as above), allowing you to write to the cells of a single sheet (an Excel spreadsheet is a Workbook containing one or more Worksheets).
  • addFormat() - returns an instance of Spreadsheet_Excel_Writer_Format, which is used to add the visual formatting of cells in a Worksheet.

The library contains three other classes of which you should be aware, although you may not find yourself having to work with them directly:

  • Spreadsheet_Excel_Writer_Validator makes it possible to add cell validation rules. Right now, there’s basically no documentation for this class. It seems to be experimental code, so I’ll be avoiding it here. Basically, it appears to provide the ability to perform basic validation on data entered into an Excel cell by an end user. More complex rules, such as validating against a list of cells, can be implemented by extending the class. The Spreadsheet_Excel_Writer_Workbook class provides the method addValidator() to create an instance of the validation while the Spreadsheet_Excel_Writer_Worksheet allows validators to be assigned to cells with the setValidation() method.
  • Spreadsheet_Excel_Writer_Parser, which is a parser for Excel spreadsheet functions that allows you to check whether a function is valid Excel syntax. This may help you trap errors when adding functions to the spreadsheet within PHP.
  • Finally, Spreadsheet_Excel_Writer_BIFFwriter is used to generate the Binary File Format for storing Excel files. If you’re interested in Excel hacking, it may be interesting to study what it’s doing but, otherwise, the library hides you from this class completely, so you don’t need to worry about it.

Zero Index Confusion

One method of note, which we saw in the above example, is the Spreadsheet_Excel_Writer_Worksheet::write() method, which you’ll be using a lot to add data to cells. It can be slightly confusing if you’re used to the way cells are addressed in Excel.

The first argument to write() is the row number. The first row number, at the top of the spreadsheet, is 0 (zero) in PEAR::Spreadsheet_Excel_Writer, not 1, as it is in Excel.

The second argument is the column number. Now, columns in Excel are identified with letters of the alphabet, not numbers, so you’ll just have to get used to translating between the two. The letter F is 6th in the alphabet, so the second argument is… 5 (of course!) — the leftmost column is 0 (zero) in PEAR::Spreadsheet_Excel_Writer, so you need to subtract one to get the column number.

The third argument to write() is the data to put into the cell; there’s an optional forth argument, used to apply visual formatting to the cell.

There are many more methods in the Spreadsheet_Excel_Writer_Worksheet class, such as for freezing and thawing parts of the sheet, and formatting the sheet, as a whole, for printing. I’ll touch on some of these in later examples but you’ll have to explore the majority for yourself.

Adding Cell Formatting

So, how about making the spreadsheet look pretty? We can accomplish this with PEAR::Spreadsheet_Excel_Writer using the addFormat() function to fetch an object of type Spreadsheet_Excel_Writer_Format. We apply the formatting to this object using the (large number of) methods it provides, then pass it the write() method of Spreadsheet_Excel_Writer_Worksheet to assign the formatting to a particular cell we’ve added.

For the sake of a “real world” example, let’s say I want to give users of my online shop, phpPetstore.com, the ability to download a “receipt” for the items they just bought as a Workbook containing a single Worksheet.

I begin my worksheet with the usual stuff:

<?php
require_once “Spreadsheet/Excel/Writer.php”;

// Create workbook
$xls =& new Spreadsheet_Excel_Writer();

// Create worksheet
$cart =& $xls->addWorksheet(’phpPetstore’);

Next (blessed with the knowledge that I’ll only be using four columns), I’ll add a title to the sheet, merging some cells in which to place it. Here, you get your first taste of how formatting is done:

// Some text to use as a title for the worksheet
$titleText = 'phpPetstore: Receipt from ' . date('dS M Y');

// Create a format object
$titleFormat =& $xls->addFormat();

// Set the font family - Helvetica works for OpenOffice calc too…
$titleFormat->setFontFamily(’Helvetica’);

// Set the text to bold
$titleFormat->setBold();

// Set the text size
$titleFormat->setSize(’13?);

// Set the text color
$titleFormat->setColor(’navy’);

// Set the bottom border width to “thick”
$titleFormat->setBottom(2);

// Set the color of the bottom border
$titleFormat->setBottomColor(’navy’);

// Set the alignment to the special merge value
$titleFormat->setAlign(’merge’);

// Add the title to the top left cell of the worksheet,
// passing it the title string and the format object
$cart->write(0,0,$titleText,$titleFormat);

// Add three empty cells to merge with
$cart->write(0,1,”,$titleFormat);
$cart->write(0,2,”,$titleFormat);
$cart->write(0,3,”,$titleFormat);

// The row height
$cart->setRow(0,30);

// Set the column width for the first 4 columns
$cart->setColumn(0,3,15);

 

Notice first that I got the formatting object by calling addFormat() via the $xls object that represents the entire spreadsheet. I then apply some specific formatting to the object (method names like setBold() speak for themselves — see the API docs for a full list of formatting methods).

Once I’m done formatting, I call write() on the Worksheet object $cart to add to a cell, passing the formatting object as the forth argument.

One unusual thing I’ve done here is to merge four cells. By calling setAlign(’merge’) on the formatting object (normally you’d use something like ‘left’, ‘right’ or ‘center’), I’ve told Spreadsheet_Excel_Writer that it should merge all cells that have this formatting applied to them. This is why I created three empty cells and applied the formatting to them.

Using setRow() allows me to modify the height of the row, making it bigger than the default Excel row height. This method has further optional formatting arguments that allow you, for example, to apply a format object to an entire row. Likewise, for setColumn(), I can set the column width and optionally apply further formatting. The difference is that setRow() applies to a single row only, while setColumn() is applied to a range of columns.

So far, so good. Now, I need some data to add to the sheet. To avoid complicating the example (by involving a database), I’ll use an indexed array of associative arrays, which we can pretend is the result of an SQL select:

$items = array (
array( 'description'=>'Parrot' ,'price'=>34.0, 'quantity'=>1),
array( 'description'=>'Snake' ,'price'=>16.5, 'quantity'=>2),
array( 'description'=>'Mouse' ,'price'=>1.25, 'quantity'=>10),
);

The “columns in the database” are the keys of the second order arrays; ‘description’, ‘price’ and ‘quantity’, so the next thing we need to do is to add the column headings along with an additional ‘Total’ heading that I’ll use later on:

// Set up some formatting
$colHeadingFormat =& $xls->addFormat();
$colHeadingFormat->setBold();
$colHeadingFormat->setFontFamily('Helvetica');
$colHeadingFormat->setBold();
$colHeadingFormat->setSize('10');
$colHeadingFormat->setAlign('center');

// An array with the data for the column headings
$colNames = array(’Item’,’Price($)’,’Quantity’,’Total’);

// Add all the column headings with a single call
// leaving a blank row to look nicer
$cart->writeRow(2,0,$colNames,$colHeadingFormat);

You’ve already seen the formatting. What you haven’t seen before is the writeRow() method. This essentially does the same thing as write(), but allows you to insert an array of data, from left to right, beginning at the specified row and column number. It makes a handy short cut to reduce lines of code.

One further thing I want to do is to make sure the column headings will always be visible as we scroll through the list of items. In Excel, this is possible by “freezing” a “pane” — selecting a block of cells that will “hover” while a user scrolls through the data, allowing them to see the column headings (in this case) that tell them what the data represents. The same is possible with PEAR::Spreadsheet_Excel_Writer:

// The cell group to freeze
// 1st Argument - vertical split position
// 2st Argument - horizontal split position (0 = no horizontal split)
// 3st Argument - topmost visible row below the vertical split
// 4th Argument - leftmost visible column after the horizontal split
$freeze = array(3,0,4,0);

// Freeze those cells!
$cart->freezePanes($freeze);

Note that “freezing” was applied directly via the Worksheet object $cart, rather than via a formatting object, because it applied to a collection of cells. Formatting, on the other hand, is applied to individual cells.

Finally, I loop through the items in the cart, adding the data to the sheet:

// Pseudo data
$items = array (
array( 'description'=>'Parrot' ,'price'=>34.0, 'quantity'=>1),
array( 'description'=>'Snake' ,'price'=>16.5, 'quantity'=>2),
array( 'description'=>'Mouse' ,'price'=>1.25, 'quantity'=>10),
);

// Use this to keep track of the current row number
$currentRow = 4;

// Loop through the data, adding it to the sheet
foreach ( $items as $item ) {
// Write each item to the sheet
$cart->writeRow($currentRow,0,$item);
$currentRow++;
}

There’s nothing particularly new here, except that you’ll find it becomes important to keep track of row and column numbers as you loop through data, because these are needed to insert the data into the correct place.

That’s basically it. If you’re new to OOP in PHP, this may, at first glance, be a little intimidating, but you’ll notice that all the methods are well named to the point where you can usually guess their purpose just by looking at them. The notion of fetching one object from another may be new, but when you think about it, it makes sense that you create a Worksheet object by calling the addWorksheetSheet() method of the Workbook object and that you add formatting objects to a cell at the point where you write() to the Worksheet.

Adding Excel Functions

Now, you’re able to make a spreadsheet that looks nice but, as any Excel Pro can tell you, simply displaying raw data isn’t very useful. Life gets really interesting when you start using Excel’s functions (and perhaps you own) to perform calculations on the raw data and turn it into something more interesting.

Now I’m not an Excel master (and this is not about to turn into an Excel tutorial) but it is clear that my shopping cart receipt needs to be cleverer, so I need to add some calculations based on the data I’ve already added. For each row, I want to display the “total item cost” — the raw data contains the unit price on the item and the number of items purchased:

"total item cost" = "unit price" * "number of items purchased"

In terms of Excel, to calculate the total for the item on the fifth row, the formula might be:

[Cell D5] =PRODUCT(B5:C5)

To accomplish this with PEAR::Spreadsheet_Excel_Writer, I need to modify slightly the code that loops through the data:

// Use this to keep track of the current row number
$currentRow = 4;

// Loop through the data, adding it to the sheet
foreach ( $items as $item ) {
// Write each item to the sheet
$cart->writeRow($currentRow,0,$item);

// Remember Excel starts counting rows from #1!
$excelRow = $currentRow + 1;

// Create a PHP string containing the formula
$formula = ‘=PRODUCT(B’ . $excelRow . ‘:C’ . $excelRow .’)’;

// Add the formula to the row
$cart->writeFormula($currentRow,3,$formula);

$currentRow++;
}

Adding the formula itself is pretty easy — we simply use the writeFormula() method. But most important (and confusing) is what I mentioned earlier — Excel begins to count rows from 1, while PEAR::Spreadsheet_Excel_Writer begins at 0 (zero), so, when creating functions, I need to remember this or I’ll be referring to the wrong cells. This is why I created the variable $excelRow, which is the $currentRow plus one. You may think this a design flaw on behalf of the authors, but remember: in PHP, like most programming languages, indexed arrays begin with a zero index. Trying to bump them forward by one just to play nice with Excel would likely have lead to many bugs and maintenance headaches. If it really annoys you, knock up some functions to translate between the two.

So, now my sheet displays the item totals on each row. But what about totaling the totals, so the customers can see the figure that will appear on their credit card bill? For this, it’s simply a matter of adding all the item totals together and displaying the result in another cell.

In Excel terms, I need to use the SUM() function to add item totals, which appear in column D:

[Grand Total Cell] =SUM(D5:D7)

To fit this into the spreadsheet, after the loop has finished, I add the following:

// The first row as Excel knows it - $currentRow was 4 at the start
$startingExcelRow = 5;

// The final row as Excel
// (which is the same as the currentRow once the loop ends)
$finalExcelRow = $currentRow;

// Excel formal to sum all the item totals to get the grand total
$gTFormula = ‘=SUM(D’.$startingExcelRow.’:D’.$finalExcelRow.’)';

// Some more formatting for the grand total cells
$gTFormat =& $xls->addFormat();
$gTFormat->setFontFamily(’Helvetica’);
$gTFormat->setBold();
$gTFormat->setTop(1); // Top border
$gTFormat->setBottom(1); // Bottom border

// Add some text plus formatting
$cart->write($currentRow,2,’Grand Total:’,$gTFormat);

// Add the grand total formula along with the format
$cart->writeFormula($currentRow,3,$gTFormula,$gTFormat);

Again it gets even more exciting, but keeping track of the Excel row numbers is mostly a case of remembering to add one to whichever variable has been tracking the PEAR::Spreadsheet_Excel_Writer row numbers. Notice also that I can apply formatting to the output produced by the formula.

Finally, I finish of my shopping cart receipt by sending the spreadsheet straight to the browser:

// Send the Spreadsheet to the browser
$xls->send("phpPetstore.xls");
$xls->close();
?>

Filename: phpPetstore.php

That’s it. The spreadsheet is ready to download. The finished code is available here.

Wrap Up

As you’ve seen, PEAR::Spreadsheet_Excel_Writer offers pretty much everything you need to build a useful spreadsheet, including formatting and functions. And, because you’re tapping into Excel’s functionality as well, you’ve got a lot of power at your disposal.

The API is tidy and, once you get used to it, easy to work with. The classes are also well structured, so scaling what I’ve done here up to a workbook that contains numerous, interrelated sheets is relatively easily accomplished. Be aware, though, that as you’ve seen here, you can end up with some pretty lengthy scripts if you’re not careful, particularly as formatting has to be defined in fine detail. If you have a need to do some serious work with PEAR::Spreadsheet_Excel_Writer it’s worth considering the opportunities for building in re-use early. You may find there’s a particular cell format that keeps cropping up all over, and could be better placed in a class and re-used. If you’re building a workbook containing many similar worksheets (e.g. Sales figures broken down with a worksheet for each region), writing classes to act as a template (design pattern hint) that generates the sheets may save a lot of effort.

Overall, PEAR::Spreadsheet_Excel_Writer is great addition to your PHP toolbox you’re your users are bugging you because they can’t get the “view” they want on the data you deliver to them with HTML, Spreadsheet_Excel_Writer provides a handy alternative to implementing a never-ending list of new features. What’s more, it creates a “wow” factor with which you can impress a potential client — particularly if the client in question uses Excel as their daily bread and butter.

The subject of PHP and Excel becomes even more interesting when you consider Jedox’s Worksheet Server, a tool for reading (here we were just writing) Excel spreadsheets, and generating PHP applications from them. But that’s a story for another time…

Get In The Google Top 10 - Post Title Optimization

Posted in General on September 4th, 2007 by Valeriu Buzila / No Comments »

Post Title Optimization is one of the most important things you need to do for getting good or great results in the SERPs. When you know how to choose a title for your post than you can get a lot of traffic from the search engines!

Optimizing Post Title Includes:

1. Choosing your title carefully - it means that you use the title that is most likely to be used for a given term
2. Use the term in your post - mention the term on the beginning of the post and in the end of your post.
3. Comment and mention the term - Respond to someone’s comment and mention the term in the comment
Recently I have done a little research on how to get in the Google Top 10 for several search terms and I’ve been more than successful.

Here’s an example: I wanted to rank in the Google’s top 10 search results for the term “add categories to blogger” which is a very often used term in the search engines. The term specified shows up 22.200.000 on search results and it’s not that easy to get in the first 10 pages of the search, not to say in the top 10!

But, by carefully choosing the post title for my post and doing some post title optimization I got into top 5 of the search results for that term! Try typing in the term “add categories to blogger” or “adding categories to blogger” or even “adding categories to blogspot” and you’ll see my blog showing up in the 5th place!

SEO Series Batch 7 - Top 5 Google Ranking Factors

Posted in General on September 4th, 2007 by Valeriu Buzila / No Comments »

SEO Series continues and the linky love is getting bigger every time. Today I will show you the most important search engine factors that Google’s algorithm known as “Googlebot” values the most. . SEOMoz released a document about the Google’s algorithm and how it works and here are top 5 positive and top 5 negative factors from that document.

Top 5 Positive Factors:

1. Keyword Use In Title Tag - In my opinion the most important factor for getting great search engine rankings!
2. Global Link Popularity Of The Site
3. Anchor Text Of Incoming Links
4. Link Popularity Within The Site
5. Age Of The Site

Top 5 Negative Factors:

1. Server Is Often Inaccessible To Googlebot
2. Content Which Is Very Similar Or Duplicate to Existing Content On The Web
3. External Links To Low Quality Sites
4. Participation In Link Schemes or Actively Selling Links
5. Duplicate Meta Tags On More Pages