Tue, 23 Sep 2008

Quick Run Down of How the Freshports Graphs Work.

The graphs I put together for Freshports are actually quite simple. To keep things simple I used jQuery and Flot to do all the heavy lifting for me. If you don't know what jQuery is I'd suggest you go read up on it. I'm certainly no javascript rockstar but the few times I've had to do things in javascript I've always used jQuery and simply love it. Flot is a graphing extension to jQuery. Using these two I was able to generate these graphs in very few lines of code all while off-loading some of the processing power needed to generate them onto the client.

The Setup.

It all starts with graphs2.php. You get both the jQuery and Flot libraries, along with a third file (graphs.js) which is where the meat of the code lives. The important piece to notice is the select element used to select the graph to be displayed. The value arguments to the option elements contain function names we will use later on to do the graphing. These value arguments are all pulled from the freshports database (which isn't how I would have wanted to do things but I had to work within the existing structure of Dan's database).

Further down the page you'll also see some div elements (title, overview, list and holder) which we will act as containers for the various pieces of the graphs we will put together. The last piece to the setup is in graphs.js itself: 

$(document).ready(function() {
    // Render the first graph.
    eval($("select option").val());
    $("select").change(function() {
        eval($("select option:selected").val());
    });
});

This block of code is what is run when the page is ready and should be obvious to anyone who knows jQuery. So that we have a graph to show when the page loads we make our first eval() call: we call the function name in the value argument of the first option element. I'll give more details about how those functions work in The Execution section. The last little bit of code sets up a function to be called whenever the select element changes. Oddly enough we call the function name contained in the value argument of the newly selected option. ;)

The Execution.

So now that you know how the various javascript functions are called let's explore what they actually do. Because each graph is different there is no single function I could write that would handle all the graphs - it would be possible if each graph were the same basic graph. Let's examine the top10committers() function (to be honest, a lot of this graphing code is munged together from the Flot examples): 

function top10committers() {
    $("#title").html("<h4>Top 10 Committers</h4>");
    $("#holder").width(800);
    $("#holder").height(500);
    $("#overview").hide();
    $("#list").hide();
    $.getJSON("generate_content.php?ds=top10Committers()", function(d1) {
        $.plot($("#holder"), d1,
            {
                bars: { show: true },
                legend: { show: false },
                xaxis: {
                    ticks: function(axis) {
                        var ret = [];
                        var i = 0;
                        for (i = 0; i < d1.length; i++) {
                            ret.push([i + .5, d1[i].label]);
                        }
                        return ret;
                    }
                }
            }
        );
    });
}

We set the html, width and height of the "title" div, and then hide the "overview" and "list" divs (this particular graph does not need those features). The next line is where the real meat of the code lives. We make a jQuery call to get a JSON object from the server. We make this call to the generate_content.php script on the server with a ds (dataset) argument of "top10Committers()". The generate_content.php script executes the necessary SQL, formats the data into a JSON object and returns it. You can see the JSON object yourself. Upon getting a response back from the server the getJSON call executes a function embedded in the call. This function is a standard Flot plot call - $.plot() - so I won't go into the details of it. If you want to read more on how Flot works there are examples, API documentation and a basic README. If you want to explore the other functions being called you can help yourself.

The generate_content.php script is a bit ugly (and luckily not visible to the user) but it is a large switch statement. For any given recognized ds value it will execute a query and generate the JSON object (by hand) to send back to the caller. There is no dynamic SQL queries going on. All of the queries are literal strings passed directly to the SQL engine. In the event that an unrecognized ds is passed to the script it simply returns an error. Flot won't know what to do with this and will refuse to draw the graph.

The Conclusion.

The end result is graphs which can easily be automatically updated by sending a new query to the server. At one point I actually had the graphs updating on a timer. It would be very easy to make these graphs auto-update like that, but I didn't think it would be useful given that a lot of the data is aggregated to a monthly or daily basis. Flot also gives us the ability to interact with the graphs (zooming, selections) with minimal effort. The code for doing this is documented in the Flot documentation so feel free to look there for more information.

The normal approach to Freshports graphs is to generate the static images on a periodic basis and simply serve the image to the browser. This approach means we can have graphs that can be updated with a single SQL query on demand and all the processing power to display the data is off-loaded to the client. There is no need for a periodic script to run to collect the data and generate the graph. It could mean a slightly higher network load as the data being requested can be bigger than the actual graph. For example the Commits Over Time By Committer graph takes a few seconds to load the data (388KB), but once it is loaded the graphing is instantaneous. Personally, I like the ability to graph multiple people at once (and yes, I'm aware of the bug where graphing too many people causes the legend to overflow the container and the graph to look bad).

So I don't know if it's a better way to generate graphs but they certainly are pretty, easy to do, and fun.

posted at: 09:23 | tags: , , | path: /entries/geek | permanent link to this entry

Sun, 10 Aug 2008

Freshports Graphs

Remember those graphs I was talking about a while back? Well, I finished them well enough to show off to people back then. In the interim I was asked if I wanted to prepare a patch to include them in Freshports. After traveling a bit, working on other things, and generally having a life I finally got around to doing that (for the record, I've only had Freshports CVS access for a week). I even fixed up some bugs in them. The patch is a single line addition, 4 new files and a handful of SQL. I'll be sending that off tomorrow night and Dan will hopefully approve of it. Once the graphs are online I'll be sure to post a link here. It's fairly easy to generate new graphs so I'm open to doing that if people have suggestions once they see what is currently available.

posted at: 23:02 | tags: , | path: /entries/geek | permanent link to this entry