I’m sure everyone and his brother will be posting a link to this article from Steve McNally’s blog on Forbes. The post contains a lot of interesting links, as do the comments.

Filed under: data, software, statistics | Leave a comment »

Posted on November 20, 2010 by John

I’m sure everyone and his brother will be posting a link to this article from Steve McNally’s blog on Forbes. The post contains a lot of interesting links, as do the comments.

Filed under: data, software, statistics | Leave a comment »

Posted on July 6, 2009 by John

I have played around with Wolfram Alpha and I get the feeling it would be very, very useful to me if only I could find the time to learn more about it. This article from MIT Technology Review provides a lot of the detail.

One thing I’ve been a bit chagrined to learn is that Alpha duplicates much of the functionality of Wolfram’s Mathematica. I just bought the home edition of the program!

Filed under: computers, data | Leave a comment »

Posted on July 3, 2009 by John

As any student of calculus knows, the *integral* of a function over a given range is closely related to the average value of the function over that range. One very useful average in studies of energy use is the daily average temperature. Building energy use is often a strong function of outdoor air temperature because about half of the energy required is for heating and air conditioning. Consequently, daily energy use in a building usually correlates well with daily average temperature.

Weather stations report temperature at regular intervals, and the best way to find the daily average is of course to take the average of all readings over a 24-hour period (assuming there are no gaps in the data). In historical studies however, sometimes the only information available is the daily high and low. But believe it or not, averaging the daily high and low temperatures gives a pretty good estimate of the daily average temperature.

Students in calculus classes are taught various methods of approximating an integral. The simplest is the rectangular rule, which in the case of daily temperature is equivalent to taking the average of all of the readings over the day (the integral would then be equal to the average multiplied by the total length of the interval, which is 24 hours).

Averaging the maximum and minimum of the function over the range gives about the same answer as the rectangular rule, but to my knowledge this method of approximating an integral is never taught. It would be interesting to know how “smooth” a curve has to be for this hold. It’s trivially true for a straight line, for example. What about second order curves?

Another interesting method of finding daily average temperature is to measure the temperature at 5:04:18 AM and 6:55:42 PM, and average the two readings. What is special about those two times? The answer is left as an exercise for the reader. Hint: it’s closely related to another method of approximating an integral, one that is occasionally still taught in classes on numerical analysis.

Filed under: data, energy, statistics | 2 Comments »

Posted on February 21, 2009 by John

The about page for Data Evolution says the blog is “dedicated to exploring the disruptive changes underway in global data markets,” and their interests seem to overlap quite a bit with my own: statistics, data visualization, R, public policy, etc.

I particularly liked the post entitled What can Darwinâ€™s finches tell us about the downturn. If you’re getting tired of the doom and gloom coming out of Washington these days, read this post. It’s one of the few things I’ve read lately that makes an attempt to look on the bright side of the current crisis.

Naturalists studying Darwin’s finches on the Galapagos Islands had found that during the wet season — when food was abundant — all birds were well-fed regardless of the length of their beak. But in 1977, an extended drought left only one in seven finches alive on the islands. Those that survived had longer beaks, which presumably gave them an advantage in finding food. “*The birds were not simply magnified by the drought: they were reformed and revised. They were changed by their dead. Their beaks were carved by their losses.*”

The post concludes:

Downturns are not only good for innovation, they are necessary. While innovation may occur in times of plenty, crises allow the right innovations (hybrid cars) to outcompete the wrong ones (SUVs). This assumes that crises are allowed to run their course (the case against bailouts), but that there are at least some survivors (the case for them).

Filed under: data, graphics, statistics | 1 Comment »