Web Development Blog

Blogger, Html scraping, JQuery and YQL

For a recent project I was working on I wanted to display some sports results on my local football club's website. The association publishes the  results weekly on its website, we also wanted to post them on our website. It would be a waste of time to type them out each week so what can we do?

First solution I thought of was HTML data scrapping the results and redisplaying them, better than using an iframe as the results were not in a format that would suit our site. It would be fairly straight forward using ASP.NET on the server but this website is a Blogger site with limited server side functionality.

Earlier I had come across Yahoo Query Language or YQL, I thought it may be possible to use YQL with jQuery on Blogger to achieve the HTML data  scrapping.

Yahoo Query Language Developer Network explains YQL or the Yahoo! Query Language is an expressive SQL-like language that lets you query, filter, and join data across Web services. Yahoo also provide a web based console that allows you build and test your YQL statements. I was pleasantly surprised at how easy it was to use. There is plenty of documentation and examples to get your on track.

Using YQL you can retrieve a HTML page from a website extract a section of the HTML using an XPath query and supply the result as either JSON or XML. I started with one of the examples in the documentation.

select * from html where url="http://www.example.com/results.html" and xpath="//table[@class=\'results\']"

I selected the JSON format as I wanted to use the resulting data with jQuery and write out the bits I wanted. In the YQL console you can test and view the results of your query. From the console you can copy the YQL url, I have replaced the domain in the example.


The url returns a page containing JSON that represents the table of results  was selecting in my query. I can use jQuery to obtain the JSON and write to my page.

<div class="results"
<script type="text/javascript"
function renderResults(data){ 
    var $table=$("<table width='80%'/>"); 
    var table; 
    else if(data.query.results.table){ 
    for (i=0;i<table.tr.length;i++) 
        var tr=table.tr[i]; 
        var $tr=$("<tr class='ResultRow'/>"); 
            for (j=0;j<tr.th.length;j++){ 
                $tr.append("<th class='ResultCell' width='60%'>"+tr.th[j].p+"</th>"); 
                $tr.append("<th class='ResultCell'>"+tr.th[j].p+"</th>"); 
            for (j=0;j<tr.td.length;j++){ 
                    $tr.append("<td class='ResultCell'>"+tr.td[j].p+"</td>"); 
                else if(tr.td[j].a){ 
                    $tr.append("<td class='ResultCell'>"+tr.td[j].a.content+"</td>"); 
                    $tr.append("<td class='ResultCell'/>") 
var url = "http://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22http%3A%2F%2Fwww.example.com%2Fresults.html%22%20and%20xpath%3D%22%2F%2Ftable%5B%40class%3D%5C'results%5C'%5D%22&format=json&callback=?"

The result produces a nice table of results which is up to date without any additional typing. This example is really only the tip of the ice berg as they say, I look forward to finding out about some of the more advance functionality.

Getting the selected value of RadioButtonList with JQuery

It struck me the other day that it is so easy to use AJAX with ASP.NET it can make you lazy. I was working on a project where I had some radio boxes and based in the selection of the radio boxes another fields was displayed or hidden. I didn't want the page to post back so initially it though I'll just pop it in a AJAX UpdatePanel and write some code to show or hide the fields after the selection is made on the RadioButtonList.

I paused for a moment and thought for a moment, its all good, its easy so why not.

I'm being lazy, I really shouldn't do a post back at all. I should write a bit of JavaScript to achieve this client side as the post back only serves to show or hide the fields.

So in the project I'm already using JQuery so its my first choice to implement my snippet of functionality.

Implementing a client side action for a RadioButtonList with JQuery

I wanted to take an action based on the selection of only one of the radio buttons. I wanted to find the selected index of the RadioButtonList. I cam up with the The functionality involved a RadioButtonList with the class 'Options' applied, which was only to save me have to get the ClientID property. If the third item is selected the FieldPanel div is shown otherwise it is hidden.

function setPanelVis(){  
        $(".Options input:radio").each(function(index) {  
            if($(this).val()==$(".Options input:radio:checked").val()){  
                    $("div.FieldPanel input").val("");  
//bind to radio button click   
$(document).ready(function() {  
    $(".Options input:radio").click(setPanelVis);  

So it works well and is all client side without any post backs. 

Using a multiple class selector on the one element

I came across the situation the other day where I wanted to apply a CSS rule when two classes appeared in the same element. The Html generated was a div containing buttons. I wanted to apply a custom background to one of the items in the list.

The Rendered Html 

As you hover over each of the button elements JavaScript toggled the button-hover class.

<div class="buttons"
    <button class="button-1">Ok</button> 
    <button class="button-2 button-hover">Cancel</button> 

I wanted to have background image change on all of the list elements when button-hover was applied. The problem was I wanted to have a custom roll over images for each of the buttons.


After a some re-reading of the CSS rules, the solution turned out to be quite straight forward. I could use the attribute value selector to test for the second class.

.buttons .button-1[class~="button-hover"
.buttons .button-2[class~="button-hover"