Sport - what are we doing to our children?

I was recently going through some of my old paperwork and I pulled out my copies of the QCS tests I took to assist in classifying my overall position when graduating high school back in 1999. I read through these tests I came across my final draft for the Writing Task.

For this task I was presented with a three-page spread of stimulus material and in two hours I had to write a 600 word document in almost any form. In this stimulus material I saw some sporting activities and some comments on tribal societies (if I ever set up my scanner I may add a copy of this stimulus material here in the future).

During school I was always the math and computer guy and writing was definitely not my strong suit. However, I feel I managed to put together a fairly decent argumentative magazine article on the topic of sport and it's relevance in today's society in this pressure-packed two hours of exam fun! It's a topic I still mostly believe in, although I intentionally made this article inflammatory to attempt to polarise and entertain the marker enough so that they may overlook the odd grammatical or spelling error.

So, without further ado, I give you the two-hour ramblings of a 17 year old some 12 years ago (with all original spelling and grammatical errors included as faithfully as possible, although I hope I made some more corrections when transcribing the final version). Oh I wish I got to come back to this a couple of days later an edit it before handing it in however, I guess everyone was in the same situation.

Sport - what are we doing to our children? 

Sport, oh what an incredibly productive activity. From birth we teach our children that the only true heroes in our society are those who participate in sport. It is the sporting events that bring the crowds, sporting heroes who gain incredible amounts of money in sponsorship and children who are forced to believe that to be famous and well-liked they must be good at sport.

What is this fascination we have about sport? Most people argue that we participate in these activities to stay fit and healthy. This is a long way from the truth. Take Gridiron for example. This game consists of very big, heavily padded men who run around a field for three hours (remembering this is a sixty minute game) tackling other men. Is this what we call keeping fit? Is this really what we want to be teaching our children? Aparently so, as parents are usually the ones who push their children into playing these sports.

From birth, the male child is continuously pressed into participating in some for of team sport. This decision is usually made by the child's father and is usually in the form of a sport the father enjoys watching. The parents often take a more active roll in the game than the children themselves: screaming from the sidelines, arguing with the referees and generally pushing their children until they have nothing more to give. This ritual of making a child do something the parent wants them to do is very damaging for the child's developing personality. Rebellious behaviour only occurs because the children finally realise that they don't have to do everything their parents want them to do anymore. They gain control over their own life.

However, this is not where this cycle ends. The rebellious child turns into an adult and, after a few short years, they have a child. Because these new parents choose to forget the reasons for their rebellious stage, the cycle of pressuring their children into playing sport continues.

This ritual like behaviour of parents pressuring their children into believing sport is the only way to win their love stems from tribal times. Earlier in history men, predominately, were the group responsible for catching the meat. They had to be fit, just in order to survive so, they made sure their children were fit also. They made them participate in 'manly' activities to build muscle and fitness so some day they could be the hunters. As humans found less and less need for hunting our food, sport became a popular past-time. Instead of hunting, games were created to expel the extra energy no longer used in hunting and thus leisure-time sport was invented.

Now, in the present, even though we know it is based on stupid ideas, sport is more popular than it ever was before. Almost every weekend, parents all over the world are forcing their children to arise early to go off and participate in contact sports, that have the potential to cause permanent injuries. that affect a person for life [ed: wow… what a long and convoluted start of a paragraph]. These ideas can't possibly be what we want for our future leaders to have. Surely the cycle has to be broken sometime. Why can't that time be now? Sport is destroying the ability of our society to live in peace and harmony together by keeping the aggressive behaviour of tribal men alive in present day children. 

Open letter to the Australian NBN2 detractors

What's the point of the National Broadband Network Version 2.0 (NBN2)? It seems simple to me, the existing copper network is aging and unmaintained. What do we replace it with?

Wireless services, on the face of it, seem like a great solution. The issue is, there is only so much bandwidth (in Hz) available through the air, a figure which is dwarfed by the quantity available via copper which is dwarfed again by that available for delivery via light. 

You may say that ADSL & wireless technologies are adequate for the average user. Right now, you're probably right. 

However, this usage pattern is going to continue to change. Mid-to-high range TVs now are being released with Internet-based connectivity options bringing streamed services like YouTube, ABC iView, radio, etc to the average user. A wireless network, even if its density was increased by a significant factor, would fall over with a few thousand video streams at 400kbps each in a single regional zone. 

This assumes providers & consumers of this content are going to be content with their low-quality versions of content. How long will it be before people start streaming 720p (> 2Mbps) and 1080p (> 4Mbps) content in 3D (x2 the bandwidth) in a way that is accessible to anyone?

Again, you may say that the existing copper-based network could handle this, but speaking as someone who lives in a suburban area (the Gold Coast), the current copper-based network isn't suitable. Not only do I not qualify for ADSL2 on Telstra or other providers' networks, I am also lucky to max out my connection to 400KB/s to Australian servers - and this speaks nothing of the pitiful upstream capabilities available to me to make use of the many technologies to allow me to either communicate (video-conferencing) or work from home (requiring large amounts of uploaded data whether it be email, documents, source code or test data).

So, given the existing infrastructure is very quickly becoming inadequate for most users, what do we replace this with?

Do we build out the wireless network, increasing it's density, all the while, running out more fiber trunks to these new towers?

Do we increase the connectivity and technology used at the copper 'node', rolling out more fiber to them along the way? Not to mention increasing the density of the nodes to decrease the copper between homes and the nodes?

Do we replace the copper network with fiber? A technology that doesn't degrade at anywhere near the pace of copper cable. That continues to provide bandwidth capabilities that outstrip demand (1Gbps downstream and 100Mbps+ upstream available now for the last-mile connectivity - 40Gbps x 64 DWMP [with 100Gbps coming soon] over a single pair of fiber cables with regenerators throughout the network that aren't specific to the speeds being pumped through them using some interesting tricks of physics to increase single power). 

Sure, its a lot of money (or maybe it isn't... they haven't actually priced anything yet). But it is truly a future-proof option. We live in a world that is starting to provide access to higher-bandwidth content direct to its lay-man consumers. 

More and more bogans are going to demand access to their HD on-demand content to their living room TV and to their kids TVs whilst they download the latest magazines to their iPads, videoconferencing in HD to their relatives & friends. And they are going to do it whilst demanding that there be no slow-downs, huge usage bills and whilst their neighbours all do the same.

Efficiency of HTTP Push Vs Pull

I'm working on a new application (StaffLocation.com) that requires the ability to stream live data from a server to the client web application via JavaScript. There are two methods that I could approach this.

The first (and simplest) involves polling a script on the server every x-number of seconds to see if the data has changed. This is the traditional way most web applications retrieve status updates from the server but it has two nasty side affects:

1. Every active client sends a request to the server every x-seconds whether there is changed data or not. Therefore, if there are 1,000 users on the website and you want the data to be updated every second, then those users will generate 1,000 hits to your application back-end (and that's 1,000 times whatever database reads/writes you do per hit).
2. Every hit uses bandwidth whether it contains data or not. A blank request/response will still contain many hundreds of bytes. Therefore, using the example of 1,000 active users polling every second at a 200 byte payload we would be using 200KB/s (1.2mbps - almost 500GB per month) of bandwidth.

Of course this situation could be improved by decreasing the frequency of polls, and some empirical testing with end-users of our prototypes show that anything below 4 seconds for an update feels instantaneous. However, bandwidth for this type of solution is still using 300kbps (125GB per month) for 1,000 users. Also, advanced caching techniques will be required as there is no way MySQL can handle the authorization & status checks required for much more than 1,000 hits per second. Maybe it's time to investigate other options.

The other type of client/server communication is via server-push. So, instead of the client requesting the server for a new status with 90% of the responses being, "NOT YET!!" The client connects once to the server and the server sends the clients updates on what the client is listening for. At this point I hear you all telling me that HTTP and the web just doesn't work this way. It is a PULL-only based technology. Well, some of you might be surprised to know that server-PUSH functionality has existed since Netscape 1.1 in the form of the content-type multipart/x-mixed-replace. The trouble is, Internet Explorer doesn't seem to support this content-type anymore (it did in 3.0) and it is very hard to use with JavaScript. However, there are ways in which JavaScript may be pushed to the client using modern browsers.

Now, the trouble with most web servers is (especially when utilizing dynamic content creation) that they don't really like pushing data to the browser without caching it first (large static files are the exception). Luckily for me, it is relatively easy today to roll your own web server. For this proof of concept I shall be using Ruby and WEBrick.

Of course, WEBrick by default follows the same pattern of, "lets cache all the data before sending it to the client." Luckily, due to the object-oriented nature of Ruby, that assumption is very easy to override. As a proof of concept, I've created a web server that dynamically adds lines to the page every 3 seconds with the use of JavaScript. One of the tricky things to remember is that the browser is expecting regular data. If it doesn't receive it within it's timeout window (usually 60 seconds), then it will think the connection has been closed. This example sends a single space character every second as a KeepAlive packet. The code is a little long, so you can download it here (push_server.rb).

Now, of course, this doesn't solve the problem of working out when there are new statuses to update the client with, but it does provide an example of a highly efficient server that can easily handle tens of thousands of connections (with the correct file-descriptor permissions on your server) with very little load when the data-set isn't changing. Also, it reduces the bandwidth usage down to the KeepAlive packets (1 byte per 10 seconds for 1,000 users is 12.5kbps or 250MB per month).

To start the server just run ruby push_server.rb. The server will start on port 2000 and you can access the example page via http://localhost:2000/hold.

The next article will be about putting a scalable observer/listener layer on top of this HTTP push server connected to our back-end status database.

Ruby on Rails - ActiveRecord#build_from_xml function

I was playing with the new to_xml feature of Ruby on Rails and I found myself wondering... if you can create XML from ActiveRecord objects, why can't you create ActiveRecord objects from XML?

After searching for a while in the RoR Documentation I wasn't able to find the inverse functionality of to_xml. So now, it seems, I have an opportunity to contribute back to the Rails community with an a functional improvement of my own. I announce to you the build_from_xml method to ActiveRecord.

Just place the below code in your config/environment.rb file.

  require "rexml/document"
  module ActiveRecord
    class Base
      def self.build_from_xml(xml)
        xml = REXML::Document.new(xml) if xml.class == String

        ar = self.new
        xml.elements[1].elements.each do | ele |
          sym = ele.name.underscore.to_sym

          # An association
          if ele.has_elements?
            klass = self.reflect_on_association(sym).klass
            ar.__send__(sym) << klass.build_from_xml(ele)

          # An attribute
          else
            ar[sym] = ele.text
          end
        end

        return ar
      end
    end
  end

You can call this from the main class of any ActiveRecord object. Here is an example.

This ruby code:

  firm_xml = File.new("firm_data.xml").read
  firm = Firm.build_from_xml(firm_xml)

Will convert this XML file into a fully functional ActiveRecord object, including the associations.

  <firm>
    <rating type="integer">1</rating>
    <name>37signals</name>
    <clients>
      <client>
        <rating type="integer">1</rating>
        <name>Summit</name>
        <id type="integer">1</id>
        <firm-id type="integer">1</firm-id>
      </client>
      <client>
        <rating type="integer">1</rating>
        <name>Microsoft</name>
        <id type="integer">2</id>
        <firm-id type="integer">1</firm-id>
      </client>
    </clients>
    <accounts>
      <account>
        <id type="integer">1</id>
        <firm-id type="integer">1</firm-id>
        <credit-limit type="integer">50</credit-limit>
      </account>
    </accounts>
    <id type="integer">1</id>
  </firm>

You may have noticed one caveat. This function accepts well formed XML code only that conforms to your model. If it doesn't, it may produce unpredictable results but will probably raise the usual ActiveRecord exceptions in most non-trivial error cases. Oh, and it requires REXML, but you knew that already right.

I will probably convert this to a plugin in the not-to-distant future. That is if the code isn't included in Rails' release branch (hint, hint).

Good vs Great software developers.

I was recently asked an interesting question as to whether I perceive myself to be a great developer. As someone who's previously been responsible for employing technical staff, here are my opinions on the differences between good developers and absolutely great developers

• Language is unimportant:

  Programming languages are all very much the same and have some common roots.

  A good programmer will learn a number of these languages to allow him to be more flexible for his employers.

  A great programmer will learn and understand the root language and semantics (like Latin for most European languages) and be able to adjust quickly to other languages as his employer requires.

• Continual self development:

  The IT industry is a fast-paced beast. Ruby on Rails was only released in 2005 yet it has revolutionised the way powerful web-based applications can be developed.

  A good developer will stay abreast of news and technologies that most affect him (usually within the technologies he's already familiar with).

  A great developer will keep informed about as much as possible on as much as possible. Not just IT-related news, but he needs to know what's happening in the rest of the world. A great developer will also continually evaluate and test how these technologies can be used to solve the issues he's faced with on a day-to-day basis. Agility is the great developer's best friend.

• Problem solving:

  There is so much more to creating applications than writing code. Most problems don't lend themselves to being easily mapped to solutions. Of course there are usually many different solutions to the same problem, some are more correct/elegant/efficient than others.

  A good developer will break a large problem into many parts and solve each part individually following a set of standards/rules.

  A great developer will also do this, but be able to take into account the complete problem-set when deciding on a solutions to the constituent parts. Also, due to the great developers large knowledge base, solutions tend to be more correct/elegant/efficient than those of the good developer.

• Social/communication skills:

  Developers (especially web-based developers) are ultimately developing applications to be used by people.

  A good developer understands this and has empathy for the end user when developing.

  A great developer will involve the user in his design and development process. He will stay in constant contact with the end-user and attempt to provide as much of the core value required from the application as quickly as possible.

Well, that's just my opinion of course. I'm sure I've missed a number of qualities others may feel are more important than these, or listed some others don't agree with. If you're a great developer, why not drop me a line and we can chat over some a hot pot of ruby code.