Loosely Coupled

We're getting close to (finally) deploying our JCAPS rewrite to production. Our final task is to load test and "tune" the application - making sure it will handle the expected production transaction load. Prior to executing the load test, we added logging statements to report on the amount of processing time spent in each JCD (or processing unit for those non-JCAPpers out there). Using this info, we can determine the slow running processes and optimize them. For this optimization we could follow a traditional approach and profile the code, or we could take an easier route and configure JCAPS to throw more resources at the bottlenecks. This second option is done in the JCAPS connectivity map by increasing the maximum number of threads allocated to run the process. Clicking the input line to the JCD to open the properties, you can set the "Server session pool size" in the Advanced tab (NOTE: this setting is only for JCDs listening to JMS queues or topics). I...

When I first joined this SOA team, I questioned the architecture we were implementing. The application flow is basically linear and uses JMS to pass messages between the various functional components. The components themselves were not designed generic enough to be shared by multiple applications. Like I mentioned earlier , the main driver for rewriting this application was for speed. The whole thing seemed very unSOA-like to me. Since none of the components were reusable outside this application, why not code the entire project in a "traditional" way? This approach would improve performance, replacing the overhead of multiple marshal/unmarshal operations needed for JMS with direct function calls. My perspective changed (a little) recently when we needed to add new functionality to the system. We could have modified one of the existing components to implement the new feature. Functionally, it made sense to keep this related logic together in a single module. The main p...

When developing Java RPC-style web services, I'm torn whether to follow a top-down or bottom-up approach. Using the bottom-up approach, you create your Java classes first and use the @ WebService and @ WebMethod annotations to specify that this is your web service interface. The WSDL for the service is generated automatically during the build (pretty nice). This approach has it's advantages in that you never have to leave Java to create the WSDL. Creating a WSDL file is not trivial and using this approach means you never need to think about the WSDL's <service> , <binding> , or <portType> elements and how they're linked together. Conversely, a top-down approach means you create your WSDL first, then use a utility (either your IDE or a command line program) to generate the Java skeleton. Using this approach you need modify the generated Java stubs to call your business logic. If the interface (the WSDL) changes, you'll need to regenerate t...

I wish I was on board at the very beginning of the project I'm working on. Maybe it would shed some light on the direction the client is currently taking. This project is a rewrite of an existing, poorly performing, data processing system. The rewrite is being done in two parts. Phase 1 replaces the front half of the system - responsible for receiving messages, routing the transactions to the appropriate business logic, then finally, transmitting a response. Analysis (performed before I got here) identified this logic as the application bottleneck. This portion of the application was rewritten in JCAPS and it's currently being tested (I haven't seen any performance data for the rewrite yet). The second phase of the project replaces the business logic, currently written in C++, with BPEL . Let me say that again... the second phase of this project replaces compiled C++ with interpreted XML . For speed. Anyone else confused? Now maybe implementing the business...

We've had some interesting SOA desing questions lately. Here's a quick rundown with my thoughts... Is it a good idea to wrap DB calls in a web service? Some argue that this is a waste for internal applications. That these services don't provide any business value - query results are simply regurgitated back to the caller. While I agree that these calls don't add much business value, the benefit lies in abstracting the DB calls from the caller. In simplest terms, wrapping access means that callers need not concern themselves with managing database connections. Additional changes/enhancements to the schema can now be handled in a single place rather than mandated enhancements to DB clients - even the DB implementation, location, and authentication updates are transparent to users. To me, services like this - services that simplify interactions - are a big part of what SOA is all about. As SOA systems evolve/extend, is it a better idea to plug in new functionality or...

In another discussion recently about SOA , it still surprises me a little that people, technical people, don't get that there's not much new here. For years it's been standard practice to separate functionality into functions, classes, and modules. The idea has always been that these smaller, highly specialized components are easier to share and maintain than monolithic blocks of code. Functionally, SOA is not much different. The goals are the same - reusability and easy maintenance. The biggest difference - in the case of a web service SOA - is that the shared library included in your application is replaced with an HTTP call. If you're already supporting message-oriented applications using MQ or JMS , these messages are now HTTP requests. The process of decomposing the application into reusable services is essentially the same. The real point here is, if you already have a modularized architecture and you're having problems with application crashes, a swi...

I know I need to let this go, but I can't stop thinking about the nonsense of this BPMN -> JCAPS -> .Net web service architecture proposed by management at the client I'm working with. The main reason for the base web services to be implemented in .Net rather than directly in JCAPS is that this organization has 10x as many .Net developers as it has Java/JCAPS developers. The thinking is that this ratio will make it easier to find available bodies to maintain and enhance these services. While this makes sense, any guess on how many people know BPMN? A small handful... all of them contractors. There's not even a BPMN modeling tool in place at the company. Yet they're convinced this is the way to go. Is it reasonable to expect business users to create BPMN models? While I realize the GUI interface makes it less like "coding", I think that it'd be helpful to have a basic understanding of boolean logic, parallelism, and exception processing in cre...

I'm wondering if my company is ready for SOA . Right now the various business units in the company support many pieces of custom built technology which provide essentially the same functionality. Consolidating this functionality into a common, shared set of services is what SOA is all about right? Not only will the footprint of what we need to support be smaller, but we'll be able to meet future business needs faster (getting a jump start by reusing these prebuilt, prepackaged, pretested functionality). Isn't SOA a no brainer in this instance? I'm not so sure. Building successful SOA components (even ones designed for internal use) need a product development approach. Code before SOA is essentially "set it and forget it". Failures in a particular component have a relatively small impact. In a SOA solution, things like scalability, stability, support, and release management are much more important. The significance of these traits increase proportionat...

The JCAPS training I attended ( Foundations of Java CAPS II ) was excellent. We screamed through the course material in 3 days and had discussions on other JCAPS topics the other 2 days. Here were my impressions. I enjoyed learning about the eInsight Business Process Manager . The tools and steps involved in creating a business process are very similar to those used to create a JCD. There are a lot of interesting constructs like correlation and the "flow" element, which easily enable parallel processing. One frequent topic of discussion, was when to use an business process (eInsight) rather than a JCD. There's a certain flexibility to creating a business process, gained at the cost of speed. Since you can do virtually everything in a JCD that can be done by a business process, we questioned when each was a better fit. We didn't reach a consensus and I'm sure we'll be talking about this again. We explored the shortcomings of the repository and the be...

In recent SOA discussions with people at work, the topic of orchestration often comes up. The problem is, like Web 2.0 before it, the meaning of what an orchestration (or orchestration layer) is differs depending on who you're talking with. In my view, an orchestration simply refers to the process of integrating disparate applications. This idea is obviously not new. It's just a new name for an RPC or ESB-layer but relies on HTTP for communication instead of CORBA or something like JMS . This view is supported on Orchestration's Wikipedia entry which describes an orchestration as the process of coordinating an exchange of information through web service interactions. Speaking of the Enterprise Service Bus, it seems there is some confusion whether the "orchestration" lives at the same layer of the ESB or at a layer above the ESB and helps to coordinate ESB communications. I fall into the first camp, believing that "orchestration" is just a new ...

Again and again the topic comes up at work about fine vs. coarse grained web services. Is anyone else talking about this? I guess I don't understand what the big debate is about. I think of a fine grained service as implementing a very specific, atomic operation. Course grained services implement larger, multistep operations... possibly consuming other services. The way I look at it is this: a business has some piece of functionality it wants to make available for others to use (the service). It shouldn't matter if the steps performed by the service can be broken into smaller components. If no one can use these intermediate (finer-grained) components, the energy spent defining, building, and maintaining this (unused) functionality is wasted. My position is that it's better to wait. Should the need to reuse some of the functionality arise, then refactor the common functionality into a smaller, reusable service. Am I thinking about this wrong? What am I missing?

I'm back at JCAPS training this week. This time the focus is on the eInsight Business Process Manager . eInsight is the JCAPS architecture layer used to combine web services. The services are connected using BPEL which is an XML-based language to define business processes. The latest version of NetBeans also includes a BPEL tool, and I'm anxious to see how they compare. We have some time scheduled at the end of the formal training to explore some of the JCAPS problems we've been experiencing. In addition to the problems I've already outlined with the repository . I'm hoping we can also explore the following: Lately, we've seen some weird behavior where things that were previously working stopped. The errors in the logs don't seem to make any sense. At first we thought it might be due to some code that was copied to a different location, but I've seen this problem where no code was copied. The only remedy we've found is to recreate both the...

I haven't had much chance to post since starting my new position. Getting up to speed in my new role has kept me busy. I have been using JCAPS though and thought I'd post a quick update on my impressions. Basically the good is still the good, but my team and I are experiencing more pain points, mostly due to the poor repository integrated into the tool. Here's a quick list of things I'd like to see improved. I've gotten in the habit of diff'ing my changes against the repository version to ensure I'm only checking in changes I've intended. JCAPS does not allow you to do this. There's also no easy way to diff changes between different repository versions. You need to be creative, and go outside to tool for this functionality. There's no easy way to find all the files you have checked out of the repository. You need to manually keep track of the files you've touched. I've become accustomed to relying on my development tools to ke...

I've been in JCAPS training all week, and so far my exposure to the product has produced mixed feelings .... the good.... JCAPS is definitely meant for enterprise-class application development. The architecture makes connecting to files, messaging queues, and databases trivial. It's scary easy. Things like monitoring and deploying to multiple environments are baked in. Again, it's nice to not have to worry about developing or searching for tools to manage these tasks. The JCAPS tools also produce some nice pseudo-documentation (diagrams and high level descriptions depicting what the application is doing and how things are connected). Some may think this is a little cheesy, but I like the idea of getting a high level picture of what is going on without having to surf through multiple source and configuration files. Having some built in transparency is nice (especially in large systems). the bad.... Is also the UGLY ... The JCAPS is built on NetBeans 3, whic...

In the midst of a job change, I've been spending most of the last few weeks handing my work off to others and writing some final documentation. With that almost complete, I'm turning my attention to my new position where I'll be working to integrate systems together in a Service Oriented Architecture (SOA). While I have SOA experience (creating & integrating web services as well as CORBA modules) my new company has chosen the Java Composite Application Suite (JCAPS) as it's SOA platform. It's (by far) the most comprehensive SOA solution I've had opportunity to work with. I'm really excited to get started. Here are some things I really like about the architecture and questions I have. The system is built on an application server and includes things like messaging, as well as consolidated user management and role based access built in. This is a big change from my current situation where I've needed to integrate (and implement) a lot of this f...

In my last post, I described I was implementing a SOAP web service controller between my Ruby chat bot and a CORBA component my company is developing. The WDSL for this service was straightforward, and with help from the Eclipse Web Service wizard, it was complete in no time. I had some downtime before the CORBA module I needed to connect with was complete, so I began to look into re-implementing my controller as a REST web service. A little digging led me to Restlet . Restlet is a lightweight framework for developing REST web services in Java. I've seldom had an easier time integrating a new Java package and getting something up and running. Restlet comes bundled with it's own HTTP server, but I opted to deploy a WAR file to our existing Tomcat installation. There are three basic steps needed to get this to work. Creating a Resource is the first step. The Resource is the brains of your service. It captures request parameters (from the URL or encoded in the body), the...

I've spent a lot of time away from the office lately and haven't had much of a chance to post. Here's a quick update on what I've been thinking about. First off, the Ruby XMPP-bot I had started is complete. This was insanely easy to implement in Ruby. The only snag I ran into was in creating a new chat room with the bot. The 0.3.1 release of XMPP4R seems to hang when creating a new room. After some research , it was apparent that others experienced this problem and had already submitted a patch. Cool. My next job is to connect the bot to an application my company developed. Communication to this module has been done previously using CORBA. Unfortunately (or not), I have not been able to find an acceptable Ruby CORBA module ( Rinn looks like it's dead and R2CORBA does not appear mature enough). Recently many of our legacy CORBA API's have been replaced with Web Service API's and I'm going to take this approach with this module. While researchi...

I'm a big fan of Web Services and Service Oriented Architectures (SOA). I've developed a handful of web services for my company to use internally and there are a few reasons I enjoy working with this technology. Ease What I think I like best, is how easy it is to create a web service. I use Eclipse and this IDE provides a web service wizard (I'm sure other tools have something similar). All you need to do is create the WSDL and most of the heavy lifting is done for you (opening and closing connections, serializing the object to the wire, etc). Hook the generated code into your business objects and it's ready to go. Since Web Services use HTTP, installing your WAR file on the web server is the final step. No other setup is needed. I did not find CORBA as easy to use. In CORBA, you need to communicate what ports you want to use, optionally set up a CORBA name service, etc. If there was a port conflict on one of the machines you were using, you needed to ...