Loosely Coupled

I started this podcast project so I could learn some things I haven't had time to experiment with in my "real" job. Things like playing more with Ruby, trying out Behavior-Driven Development, and experimenting with Agile practices (plus I really wanted a better podcatcher program and I needed something to do over the winter). While I haven't produced much real code yet, I've created some simple prototypes and wanted to comment on some things I've been looking at. Reading an RSS feed My first prototype was a very simple Ruby program to read and parse an RSS feed. I had found some code here using the standard Ruby RSS Parser so that's where I started. At first I thought it was broken, but after about a minute and a half it returned. This wasn't going to cut it, so I started to look for alternatives and quickly found the feed-normalizer gem. Some quick coding and the total time to access the RSS feed was reduced to about 10 seconds. Not bad...

Making resolutions is tradition on New Year's Eve.... evaluate areas you'd like to improve (or new skills you'd like to acquire), and set goals to improve in those areas. Here are two work-related resolutions I'm going to focus on entering 2008. Make JCAPS Usable If you've been following my posts, you're aware that I've been frustrated with Sun's JCAPS product. The tool is simply not ready for prime time and it's hurting my productivity. In addition to the problems I've already outlined , developing in JCAPS' eDesigner makes me "feel dirty". You need to go outside to tool to create modules of shared code (jars). This limitation makes it hard to implement classes that follow even the simplest design patterns or integrate with third party frameworks, like Spring . I feel myself falling into bad coding habits. I'd like to find a compromise. What I'd like to do, is move my code out of the JCAPS JCD and into a sep...

As the name suggests WebDevRadio is a web development podcast. I'm a big fan of this show. It always delivers an interesting perspective, and the discussions often relate to a technology I'm also exploring. In a recent episode, the host Michael Kimsal interviewed Jason Rudolph about Jason's new book Getting Started with Grails . I've written before about Ruby and Rails and using something like JRuby to leverage Java's stability and scalability in a production environment. Grails (and Groovy ) is an interesting alternative to the Rails/Ruby/JRuby I'd been thinking about. Like Ruby, Groovy is an interpreted language but with some constructs that make it easier for Java developers to transition over. Also, since it's written for the JVM, it gets all the Java maturity "for free" and naturally integrates with existing Java modules (No need for something like JRuby). Grails brings the Rails goodness to the Groovy platform... including a lot of ...

I've spent a lot of time away from the office lately and haven't had much of a chance to post. Here's a quick update on what I've been thinking about. First off, the Ruby XMPP-bot I had started is complete. This was insanely easy to implement in Ruby. The only snag I ran into was in creating a new chat room with the bot. The 0.3.1 release of XMPP4R seems to hang when creating a new room. After some research , it was apparent that others experienced this problem and had already submitted a patch. Cool. My next job is to connect the bot to an application my company developed. Communication to this module has been done previously using CORBA. Unfortunately (or not), I have not been able to find an acceptable Ruby CORBA module ( Rinn looks like it's dead and R2CORBA does not appear mature enough). Recently many of our legacy CORBA API's have been replaced with Web Service API's and I'm going to take this approach with this module. While researchi...

The Hibernate tweaks I made a few weeks ago made our application "fast enough" for now. So I'm back off of Rails and I've started work on a XMPP (Jabber) bot to monitor chat channels. For those unfamiliar, XMPP is an open protocol for instant messaging. I've just gotten started and, again with Ruby, my progress has been quick and easy. After installing the Openfire XMPP server and XMPP4R Ruby Gem, I was off and running. With help from someone who's done this before, I was able to communicate with the server very easily. I'll be sure to post progress and I get deeper into this. By the way, if anyone's been following my Rails progress, I'm sure I'll be back to finish this at some point. As I've written before , the database design is flawed and will not scale well. We've just bought a little more time before these issues bubble up again.

Thankfully, I'm done working with Hibernate for a while and I'm back on Rails. Since I posted on FRail :include , it seems there has been a lot of plugin development to add this functionality. While all of these plugins help to address the problem, I'm having a hard time finding one to address my specific scenario. I think I've explained my situation before, but here it is again in a nutshell. The company I work for aggregates a lot of data. The goal of my application is to allow users to sift out the information they don't need, discovering nuggets of truly useful knowledge. The approach we're taking is to have users to start from something they're interested in, and use that object to find other objects with similar attributes and characteristics. What I really need from Rails is a robust way to join a lot of tables together and specify search criteria. I've decided to roll my own solution for now. What I'm developing in Rails will take in o...

I really like developing with Rails, but I've been frustrated the last few days by some things Rails should do quite easily, yet doesn't. I've already written here and here about some of my struggles with Rails :include clause. Here's one more... Let's say you have a has-and-belongs-to-many relationship. If you :include this relationship in a find query, Rails will automatically create the join clause including the lookup table! That's cool. Unfortunately, it looks as if only the :include clause provides this piece of functionality. Why isn't the same functionality provided via the :join clause or through a general purpose module? This doesn't make any sense to me! I'm working on a database application where I don't need to display a lot of data, but the data displayed is heavily filtered by it's connections to other information in the database. I need to join a lot of tables to perform the filtering operation and using :include ...

Now that the models are complete, I want to expose access to them as a web service. I'm thinking REST is the way to go. A RESTful approach seems simple. It appears to solve pesky "back button" problems and session timeout issues, but I have one big question: How to fit all the information I need into the URL? The application I'm working on has A LOT of data. The main objective of the application is to allow filtering of this data down to a useful, manageable set. It wouldn't be uncommon for a user to ask for something like: Give me everything connected to Item1, Item2 and Item3, Hide anything that has to do with Item4 or Item5 In addition, give me anything that mentions Items 6, 7, or 8 and is new to the system in the past month. To put all this information in the URL is a scary proposition. In it's previous life as a Java web application, all this search criteria was built incrementally and stored in a user's session. Since the client ...

All my models are implemented in Rails and I wanted to give a quick update on how things are going. For now, I'm been using the rails views and controllers to test my code (automated tests coming soon). This has worked out great for me. Since my application only allows users to view and filter database records, I don't have a lot of validation checks, but I wanted to get a feel for the performance of pages need to display a lot of data from various tables. One page I tested performs 35 queries (including all the counts and show field commands) and takes about 5 seconds using the Webrick server (from NetBeans so I believe JRuby is being used). The development log lists the time spent in the database as less than 1 second. I'm sure performance could be better, but this is a big improvement over what we had before and the database I'm using contains more records. I'm still noticing that when you ":include" more than one model in a "find" comman...

I was out last week, but finally got my schema in a Rails friendly format and the scaffolding utility is working great! I thought I'd hit on some of the pain points I experienced during this process and how I worked through them. MySql stinks when you need to change the schema of a database containing data. To make things easy, I've been working with an empty database and using the MySQL administrator to capture the SQL. The plan was to run this SQL against existing databases to upgrade them. This process doesn't work like I'd hoped. For some columns, the SQL query fails when I try to change a column name (like changing the primary key name to the Rails default 'id'). There are no foreign key constraints on this field and it does not fail on every table where the column name must change. I can't figure out why MySQL bombs on these specific statements. Maybe it's because these table have large row counts (~1M rows)??? Luckily, I won't be creat...

I've been playing with Rails over the past few days and want to quickly express my experience so far. Creating joins, lookup tables, tuning, grouping, sorting, and setting limits are all easy. You can do something like: Actor.sum(:appearances, :joins => "INNER JOIN appearances_show_lookup asl ON asl.appearances_id = appearances.appearances_id", :group=>'asl.appearances_id', :order=>'sum(appearances) DESC', :limit=>10) which translates into SELECT m.appearances_id, sum(appearances) FROM appearances m INNER JOIN appearances_show_lookup md ON md.appearances_id=m.appearances_id GROUP BY m.appearances_id ORDER BY sum(appearances) DESC LIMIT 10; or... Actor.find(:all,include=>:appearances_type, :conditions=>"appearances_type.name='guest'", :limit=>10) to SELECT * FROM appearances LEFT OUTER JOIN appearances_type ON appearances_type.appearances_type_id = appearances.appearances_type_id WHERE...

I picked up a copy of Rails for Java Developers by Stuart Halloway and Justin Gehtland and am playing around more with Rails. The book is really good, but I really need a migration guide from Hibernate to Rails (or an existing web application to Rails). I'm looking for more information on creating Rails models that can be accessed through a complex set of filters and associations. The things I need to know are: How to create something like a 8 table Rails join? How best to represent lookup tables? What options are available to tune the queries? Can I capture the Rails query before it's sent to the DB for debugging (outside the MySQL log)? How do I specify things like limit, paging, group by, sum, distinct, etc? I'm going to reread the ActiveRecord chapter of this book again. I also have a copy of Agile Web Development with Rails on the way. Many articles I'm finding on the Internet draw the same conclusions - if your schema and object dependencies ar...

The database cleanup is finally complete! Now I need to rewrite our web app's data access code. When we first created the database layer, we used the Hibernate framework. Since then Hibernate3 has been released and an upgrade is scheduled as part of this work. We did have some pain points with Hibernate, however. For instance, when I started evaluating our DB code, I noticed some DB queries that we don't explicitly create. Whether these were the result of a poor object-relational model, lazy instantiation, or some other Hibernate functionality is unknown at this time. I've found Hibernate (or at least how we're using Hibernate) complicated to debug. I'm going to take some time and look at Ruby and Rails as an alternative. I've mentioned before that I've wanted to learn Ruby for a while and this seems like a good excuse. There is a Ruby library for Java called JRuby . The plan is to learn the JRuby Rails extensions and find out the effort to incor...

One Laptop Per Child ( OLPC ) is a movement to create inexpensive, full featured laptops to be used by children for education. The design of this machine is geared at promoting creativity and collaboration. Each system features wireless capabilities for students to connect and communicate with teachers and other students. The user interface centers on activities that children participate in to learn. This differs from a traditional UI which focuses on tasks and applications. The laptop is enclosed in a durable case with a kid friendly handle. It also includes many power saving features, including the ability to charge the battery using human power (like turning a crank or pulling a string). This is an important consideration for areas with little or no access to power. The price of the hardware is very low (around $100). In addition, much of the software for this system is being developed in an cost friendly, open source model. I'm excited about the possibilities this ...

Since my last post, I picked up a copy of SQL Tuning by Dan Tow . This is exactly the resource I've been looking for. Dan Tow has developed a process of diagramming queries, and then using these diagrams to determine an optimized query plan. I'm only half finished with the book, and I'm amazed by the results I'm already seeing. This book has given me a new perspective (and confidence) to solve the problems I described. I'll post more on my progress in a future post. Ruby For a while now, I've wanted to learn Ruby (and Rails ). I'm impressed with how much you can do with this language in a few lines of code . If nothing else, it will provide a quick and easy way to mock up prototypes. Last week, I downloaded Hackety Hack , which is a lightweight Ruby IDE. I've completed the Hackety Hack tutorial and can't wait to create more complex examples. del.icio.us I've started to (finally) place my bookmarks into del.icio.us . I thought ...