Xero API Release V2.06

Today Xero released some extra functionality to their API including JSON responses, manual journal entry, items and PDF invoices.

Of great interest to me is the ability to download PDF invoices directly from Xero which are formatted according to the chosen branding theme. This functionality will greatly decrease the time to market of my new application (still in stealth) by a few weeks and has me very excited.

My ruby library (Xeroizer) will be updated to work with these new features within the next week or so. Keep a watch on the project to keep up-to-date as changes are made and feel free to submit pull-requests for desired functionality. I usually respond within 24 hours.

A JRuby/Rails Message Driven Bean

A while ago I found this tutorial on getting an EJB message-driven bean up and running utilising JRuby and Rails at http://nodnol.org/blog/chris/entry/a_jruby_rails_message_driven. This website now appears to be down so I am reproducing it here for posterity (and my own reference).

I'm building a system which receives messages from a larger application platform and records the details in its local database. It's built with Rails and JRuby, with Apache ActiveMQ as the message broker. Using JRuby lets me use ActiveMQ's native JMS-based client, rather than speaking STOMP to the broker (as our larger platform does on the other end of the queue). My deployment platform is Glassfish, and I'm deploying the Rails app to it in the usual way with Warbler.

To begin with, I wrote a standalone script to subscribe to the JMS queue, based on the suggestion to use JMS rather than ActiveMessaging. This works well, and is very simple to deploy for development. As Shane points out, it's just a literal translation from Java to (J)Ruby. A problem is that even when configured to use a failover transport to a pair of message brokers, the script will frequently exit and need restarting. Not only that, but starting the script as a separate process means I've got two JVMs running - one for Glassfish and one for the JRuby script, and they can't share a common JDBC connection pool, so there's more to manage there.

Finally, I'd have to manage any concurrency required myself - the script is resolutely single-threaded, and if I need multiple threads to saturate the hardware, I'd need to write the thread management code as part of the script.

Since I'm already using an app server for the web part of the system, the answer seems to be to move the script into Glassfish as a message-driven Bean.

There are a few issues to be solved: I only want a single copy of the Rails environment, and I need to arrange for the relevant parts of our application to be packaged so it can be correctly deployed with the bean.

The upside is that with Rails 2.2, I can share that Rails environment among as many threads as I need, without having to synchronize access to the app - given a JDBC connection pool, Glassfish will start many bean instances to handle incoming messages.

Unfortunately there doesn't seem to be anything like Warbler to help out with packaging the app into an MDB, so it's all rather manual. Here's what I needed to do to set this up:

• Provide an app entry point which would accept a message, and run the required business logic in ruby
• Create a standard J2EE Message-driven Bean in Java, and hook it to an external ActiveMQ instance
• Spin up a single Rails instance inside the container
• Arrange for the message receipt handler to call the app entry point with the contents of the message

App entry point for messaging

This is a class method on the model the app will create as a result of receiving the message. My messages are simple YAML strings, and this method accepts that YAML directly.

That means I'm converting to a Ruby data structure from YAML in Ruby code, and it might be better to make sure it's done in Java - the messages are small enough that this isn't a big deal though.

Create an MDB

I'm using NetBeans for this, simply because of its integration with Glassfish. Everything is done with Java 5 annotations, bar the selection of activemq instead of the built in broker (which doesn't have native STOMP support).

All that's needed beyond the standard Bean that NetBeans sets up is this stanza in sun-ejb-jar.xml:

	<mdb-resource-adapter>
	<resource-adapter-mid>activemq</resource-adapter-mid>
	</mdb-resource-adapter>

view raw gistfile1.htm hosted with ❤ by GitHub

assuming the ActiveMQ RA is deployed in Glassfish as "activemq". This configuration is taken from this forum post

Rails Instance

I only want to start one Rails instance to be shared among all the bean instances. I'll create a RailsRuntime class, which loads JRuby, initialises Rails and provides a "string eval" method, and then another class, RailsRuntimeSingleton, to maintain that single instance.

Here's the RailsRuntime class:

	import org.jruby.Ruby;
	import org.jruby.RubyRuntimeAdapter;
	import org.jruby.javasupport.JavaEmbedUtils;
	import org.jruby.exceptions.RaiseException;
	import java.util.ArrayList;
	public class RailsRuntime {
	private Ruby runtime;
	private RubyRuntimeAdapter evaler;
	public RailsRuntime() {
	try {
	String jruby_home = "/usr/local/jruby";
	System.out.println("Initializing Rails with JRuby from " + jruby_home);
	System.setProperty("jruby.home", jruby_home);
	System.setProperty("jruby.base", jruby_home);
	System.setProperty("jruby.lib", jruby_home + "/lib");
	System.setProperty("jruby.script", "jruby");
	runtime = JavaEmbedUtils.initialize(new ArrayList());
	evaler = JavaEmbedUtils.newRuntimeAdapter();
	evaler.eval(runtime, "ENV['RAILS_ENV'] = 'production'; require 'META-INF/rails/config/environment.rb'");
	System.out.println("Finished initializing Rails");
	}
	catch (RaiseException re) {
	System.out.println("RaiseException: " + re.getMessage());
	throw re;
	}
	}
	public void eval(String r) {
	evaler.eval(runtime, r);
	}
	}

view raw RailsRuntime.java hosted with ❤ by GitHub

There's a hardcoded path to the JRuby home directory here, which isn't ideal - really I should just include a jruby-complete jar but I don't have the infrastructure in place yet to also include all the required gems, so I'm relying on that fixed path for now.

I do bundle up the Rails app into the jar, and so there's a fixed path available to environment.rb. Production mode is also set here.

The singleton class is very simple. A static property is guaranteed to be initialised only once, so this avoids the need for any synchronization around the RailsRuntime setup.

	public class RailsRuntimeSingleton {
	public static RailsRuntime singleton = new RailsRuntime();
	}

view raw RailsRuntimeSingleton.java hosted with ❤ by GitHub

Call the app when we receive a message

Now I've got a Rails instance spun up, and messages coming into the bean, it's time to plug it all together. Here's the bean's onMessage method and the hook into the Rails app:

	public void onMessage(Message message) {
	TextMessage tm = (TextMessage)message;
	try {
	createEvent(tm.getText());
	}
	catch (JMSException e) {
	System.out.println("JMSException on getText()");
	}
	}
	private void createEvent(String yaml) {
	RailsRuntime runtime = RailsRuntimeSingleton.singleton;
	// no synchronize required!
	runtime.eval("Event.create_from_yaml('"+ yaml + "')");
	}

view raw gistfile1.java hosted with ❤ by GitHub

Before Rails 2.2, I needed to synchronize around the eval() method, to serialise access to Rails. Now that's not necessary, and I can allow the container to spawn as many threads as I have connections available in the JDBC pool.

My test environment is a simple, single-threaded Perl STOMP sender running flat out to an ActiveMQ instance, plus the bean in Glassfish all on a Macbook Pro. When the bean runs serialized, the receiver isn't able to keep up and a backlog of messages builds up. Simply removing the synchronized block means the receiver can keep up with the sender easily.

Next steps

There's a lot to get right with this setup even leaving aside the dependency on the external JRuby.

• The message queue configuration in Java in the Bean class
• The external message broker configuration in sun-ejb-jar.xml
• Packaging up Rails and the application as part of the Bean

More work is required to include jruby-complete.jar and remove that external dependency. Beyond that, making the Java parts generic, allowing configuration in Ruby and building outside of the IDE in the style of Warbler seems to be the way to go.

Dealing with codes - phonetic_alphabet plugin (Ruby on Rails)

Most fonts are designed for readability of words, not complicated codes and recently, we've had to provide referreral codes to our partners so that we can track the applications that come from them. Instead of sufferring through referral codes being incorrectly provided because someone has confused an O for an 0 or a 1 for a l or even an I (see what I mean!), I created a plugin that turns any string into it's spoken version using the NATO phonetic alphabet.

To install this plugin in Ruby on Rails, just type the following from your application's root directory:

script/plugin install _
https://svn01.allmyfunds.com.au/svn/public/plugins/phonetic_alphabet

To use, see this except from the README file:

You can get the phonetic equivalent of a single character with PhoneticAlphabet[single_character]

or

You can convert a string to it's phonetic equivalent using the String#to_phonetic overridden method.

  Example:

    puts "hello world".to_phonetic

  Outputs:

    hotel echo lima lima oscar SPACE whiskey oscar romeo lima delta

You can also pass in a different separator:

  Example:

    puts "hello world".to_phonetic("-")

  Outputs:

    hotel-echo-lima-lima-oscar-SPACE-whiskey-oscar-romeo-lima-delta

ActiveRecord and programmatically working with attributes.

Ruby on Rails is great. Partially because of all the ORM stuff and partially because Ruby is infitely extendable and overrideable (yes, I just made that word up).

On my current project, I have a lot of extended attributes in models that modify other attributes or even other models entirely. So, when I create new objects and assign their attributes, I would prefer the mass assignments to go through these custom mutators.

Now in Ruby, this is relatively simple. When you want to call a method within an object programmatically, you can use the #send method:

    str = "Hello world!"
    puts str.send(:size)

    Displays: 12

However, when you are performing property setting, you have to do some manipulation to the attribute name first before you can assign a value to it. For example, the code for sending the values of a hash to the corresponding mutator methods in a Person object would be the following:

    hash = {
        :first_name => "Wayne", 
        :last_name => "Robinson", 
        :date_of_birth => "1982-02-15"
    }

    hash.each do | key, value |
      person.send("#{key}=", value)
    end

As you can see, there is a little bit of repetition there if you have to use that in more than one place. So, I've DRYed this code into the following ActiveRecord extension. Just pop this active_record_getters_and_setters.rb file in your lib/ directory and require it in your environment.rb file with:

    require 'active_record_getters_and_setters.rb' 

You will now have three extra methods accessible to you in all your ActiveRecord objects.

ActiveRecord::Base#set(attribute, value)
    Assigns value to the mutator (setter) for attribute.

ActiveRecord::Base#get(attribute)
    Gets the value from the accessor (getter) for attribute.

ActiveRecord::Base#set_attributes(attribute_hash)
   Does an ActiveRecord::Base#set for each key/value pair in attribute_hash.

Multiple Values from Scriptaculous' Autocomplete

Ever wanted to extract multiple values from a Script.aculo.us Google Suggest-like autocomplete text field? I recently did and here's how.

If you aren't aware of how to use autocomplete text fields, please read over the simple and customised demos available at Script.aculo.us. Also, a warning, this demo utilises Ruby on Rails however, with a little bit of modification, this should work using the Script.aculo.us library without Ruby on Rails.

This example will auto-populate a state and postcode based on the user's selected suburb (yes, I'm Australian).

The first step is to create a view for the page containing the autocomplete field and for the autocomplete field itself.

Controller:

def new
  # This is the main controller that will 
  # contain the autocomplete field
  @contact = Contact.new 
end

def auto_complete_for_suburb
  suburb = params[:suburb]
  @surburbs = Suburb.find_by_name(suburb, 
      :order => "name", :limit => 20)
  render :partial => "auto_complete_suburb"
end

View for the new action (assume an application.rhtml template has been created, this template must include the Prototype and Script.aculo.us javascript libraries):

<%= form_tag({:action => :create}, {:method => :post}) %>
<table>
<tr>
  <th>Name:</th>
  <td><%= text_field(:contact, :name) %></td>
</tr>
<tr>
  <th>Suburb:</th>
  <td><%= text_field_with_autcomplete(:contact, :suburb, 
    :select => "value", 
    :after_update_element => 
      "function (ele, value) {
        $("contact_state").value = 
          Ajax.Autocompleter.extract_value(value, 
            'STATE');
        $("contact_postcode").value =
          Ajax.Autocompleter.extract_value(value, 
            'POSTCODE'); }
     ") %>
  </td>
</tr>
<tr>
  <th>State:</th>
  <td><%= text_field(:contact, :state, :size => 10) %></td>
</tr>
<tr>
  <th>Postcode:</th>
  <td><%= text_field(:contact, :postcode, 
            :size => 10) %></td>
</tr>
</table>
<%= end_form_tag %>

Before we continue any further, it is worth-while defining the _auto_complete_suburb.rhtml partial.

<ul class="suburbs">
<% unless @suburbs.nil? -%>
<% @suburbs.each do | suburb | -%>
  <li>
    <%= h("#{suburb[:name]}, #{suburb[:state]}" + 
          "#{suburb[:postcode]}") %>
    <div class="value" style="display: none;">
      <%= h(suburb[:name]) %>
    </div>
    <div class="STATE" style="display: none;">
      <%= h(suburb[:state]) %>
    </div>
    <div class="POSTCODE" style="display: none;">
      <%= h(suburb[:postcode]) %>
    </div>
  </li>
<% end -%>
<% end -%>
</ul> 

The autocompleter view has three (3) hidden <div> tags which contain the extra data used by the base Autocompleter Javascript methods as well as the new one (extract_value) that will be defined below. 

You may also want to cast your eye over the suburb field definition in the new contact view as this is where most of the action is. There are two options to note:

• the :value option which specifies the class name of the element which contains the value to place in attribute (the default would be whatever is within the rendered <li> field which, as we will find out below, will contain more than just the selected suburb)
• the :after_update_element option which specifies a piece of Javascript to execute when the item is selected. You will see that this Javascript executes the Ajax.Autocompleter.extract_value function twice. This function does not exist in Script.aculo.us but is provides an easy way to extract extra values from an autocomplete list.

The Script.aculo.us Autocompleter methods conviently pass the complete contents of the <li> field to the method specified in the :after_update_element option. This allows us to extract any additional values from this data. I have created a simple addition to the Ajax.Autocompleter class below that speeds this extraction:

Additional method for Ajax.Autocompleter class. This can be declared anywhere after the inital Script.aculo.us script inclusion. For my purposes I put this at the top of my application.js file. 

Ajax.Autocompleter.extract_value = 
  function (value, className) {
    var result;

    var elements = 
      document.getElementsByClassName(className, value);
    if (elements && elements.length == 1) {
      result = elements[0].innerHTML.unescapeHTML();
    }

    return result;
  };

 So that's all there is to it. If anyone would like me to create a demo of the above code, ask me and, if I get enough requests, I'll put something together.

Ruby on Rails - ActiveRecord#build_from_xml function

I was playing with the new to_xml feature of Ruby on Rails and I found myself wondering... if you can create XML from ActiveRecord objects, why can't you create ActiveRecord objects from XML?

After searching for a while in the RoR Documentation I wasn't able to find the inverse functionality of to_xml. So now, it seems, I have an opportunity to contribute back to the Rails community with an a functional improvement of my own. I announce to you the build_from_xml method to ActiveRecord.

Just place the below code in your config/environment.rb file.

  require "rexml/document"
  module ActiveRecord
    class Base
      def self.build_from_xml(xml)
        xml = REXML::Document.new(xml) if xml.class == String

        ar = self.new
        xml.elements[1].elements.each do | ele |
          sym = ele.name.underscore.to_sym

          # An association
          if ele.has_elements?
            klass = self.reflect_on_association(sym).klass
            ar.__send__(sym) << klass.build_from_xml(ele)

          # An attribute
          else
            ar[sym] = ele.text
          end
        end

        return ar
      end
    end
  end

You can call this from the main class of any ActiveRecord object. Here is an example.

This ruby code:

  firm_xml = File.new("firm_data.xml").read
  firm = Firm.build_from_xml(firm_xml)

Will convert this XML file into a fully functional ActiveRecord object, including the associations.

  <firm>
    <rating type="integer">1</rating>
    <name>37signals</name>
    <clients>
      <client>
        <rating type="integer">1</rating>
        <name>Summit</name>
        <id type="integer">1</id>
        <firm-id type="integer">1</firm-id>
      </client>
      <client>
        <rating type="integer">1</rating>
        <name>Microsoft</name>
        <id type="integer">2</id>
        <firm-id type="integer">1</firm-id>
      </client>
    </clients>
    <accounts>
      <account>
        <id type="integer">1</id>
        <firm-id type="integer">1</firm-id>
        <credit-limit type="integer">50</credit-limit>
      </account>
    </accounts>
    <id type="integer">1</id>
  </firm>

You may have noticed one caveat. This function accepts well formed XML code only that conforms to your model. If it doesn't, it may produce unpredictable results but will probably raise the usual ActiveRecord exceptions in most non-trivial error cases. Oh, and it requires REXML, but you knew that already right.

I will probably convert this to a plugin in the not-to-distant future. That is if the code isn't included in Rails' release branch (hint, hint).

Rails is truly amazing.

I can never get over how truly amazing Ruby on Rails is. I was recently asked to create a quick and nasty demo application to showcase some of my skills. So, following the provided specs, I whipped up the following application in less than a day (of course, I spent a larger portion of that than I'd like to admit on user interface tweaks - I wish I had design talent).

Now, I'm not a graphic designer/illustrator by any means, but my coding is pretty good. You can check out the application here. It's a pretty basic file uploader with the ability to make files public. Also, the upload supports RTF and HTML in addition to plain text.