Loosely Coupled

It's been close to 2 months since my first Performance Tuning JCAPS post. Since then, we've noticed our servers running out of TCP connections under heavy load. Researching this problem, we learned that the JCAPS' JMS server makes heavy use of TCP in implementing Request/Reply queues . We approached Sun for guidance and they assured us that they've had success implementing high throughput applications using the JMS request reply solution... but how? It appears a multi-step solution is needed. First, running out of TCP connections in a scenario like ours is a JCAPS bug addressed in an ESR (110348) and rolled into JCAPS 5.1.3 Update Release 3 . We installed this update and noticed a marked improvement. Running my simple 75 user simulated test, transactions took 2521 ms to round trip (compared to the previous time of 20128ms). While this is great improvement, an average of 2.5 seconds is still really slow for this simple transaction. We reported our findings to Su...

Business transactions can vary widely in their definitions, but they can basically be classified into two types - those where I need an immediate response so I can continue with my work and those where I'm going to tell you what to do and trust that you'll get it done while I move on to something else. These classifications are otherwise known respectively as synchronous and asynchronous transactions. When designing an architecture for solving business problems, it's helpful to figure out which type of problem you're solving and then pick an appropriate technology to support it. For example, HTTP might be a good choice for synchronous applications since it is request/reply by nature. JMS might be a better fit for asynchronous solutions since these structures are typically one way. Blurring the lines is the JCAPS offering of a request/reply mechanism for JMS messaging - called appropriately enough, JMS RequestReply . A colleague called this mechanism "synchronou...

We're getting close to (finally) deploying our JCAPS rewrite to production. Our final task is to load test and "tune" the application - making sure it will handle the expected production transaction load. Prior to executing the load test, we added logging statements to report on the amount of processing time spent in each JCD (or processing unit for those non-JCAPpers out there). Using this info, we can determine the slow running processes and optimize them. For this optimization we could follow a traditional approach and profile the code, or we could take an easier route and configure JCAPS to throw more resources at the bottlenecks. This second option is done in the JCAPS connectivity map by increasing the maximum number of threads allocated to run the process. Clicking the input line to the JCD to open the properties, you can set the "Server session pool size" in the Advanced tab (NOTE: this setting is only for JCDs listening to JMS queues or topics). I...

I've started to use JCAPS 6 a little of over the past month and thought I'd share some of my initial impressions. Finally a JCAPS editor I can use! I found the previous version of the JCAPS IDE hard to develop with. It was based on Netbeans 3 and didn't have a lot of the features I had grown accustomed to using ( refactoring tools and a local history to name two). It was just plain old, clunky, and slow. The new version is based on the latest Netbeans and all the goodness is back. And it's fast. No more repository. Well, almost. JCAPS 6 has a few different modes to work in - repository based or non-repository. A repository based project allows JCAPS developers to work in essentially the same way they did with JCAPS 5 - JCDs , OTDs , connectivity maps, deployment profiles and yes - the repository. It's also the way to migrate legacy JCAPS project to the new platform - through a simple export and import between the repositories. I haven't used t...

In this week's StackOverflow podcast , Joel Spolsky describes cognitive dissonance - and it may help explain the the JCAPS love fest at my current client. This conversation happens around the 11-minute mark. I knew there had to be a name for this ...

I know I need to let this go, but I can't stop thinking about the nonsense of this BPMN -> JCAPS -> .Net web service architecture proposed by management at the client I'm working with. The main reason for the base web services to be implemented in .Net rather than directly in JCAPS is that this organization has 10x as many .Net developers as it has Java/JCAPS developers. The thinking is that this ratio will make it easier to find available bodies to maintain and enhance these services. While this makes sense, any guess on how many people know BPMN? A small handful... all of them contractors. There's not even a BPMN modeling tool in place at the company. Yet they're convinced this is the way to go. Is it reasonable to expect business users to create BPMN models? While I realize the GUI interface makes it less like "coding", I think that it'd be helpful to have a basic understanding of boolean logic, parallelism, and exception processing in cre...

Come hell or high water - this company is hell bent on including JCAPS as part of it's enterprise architecture, even though it doesn't plan to use any of the features that might set it apart from it competitors or open source alternatives. The long-term plan here is for the business users to use a BPMN-compliant tool (not the one from JCAPS) to create the business processes. These processes will integrate web service calls, creating a sort of business mashup of these services. The services will be written in .Net, not JCAPS. The only place JCAPS enters the equation is as the platform to run the BPEL generated from the BPMN. Make any sense? Not to me. Isn't a BPEL engine included in Glassfish? I'm sure this is just one of several free or low cost alternatives to execute BPEL. What I can't figure out is this company's infatuation with JCAPS. Given the shortcomings they've encountered already, why they are so intent on looking for more places to impleme...

I've noticed a trend in my current organization which I'm sure is common to many companies. I'm having a hard time defining it, but let's call it "Silver Bullet Syndrome"... where when faced with some business problem, companies immediately look for an off the self (OTS) technology solution - many times without understanding the full scope of the problem. The OTS solution is viewed as a "Silver Bullet" and will immediately solve all of the organization's woes. Now I'm not saying all off the shelf software is bad, but often times these business problems are not generic enough to be addressed by a general solution. The lack of flexibility of many OTS solutions compounds the problem, forcing companies to modify business processes or requirements to meet the capabilities of the tool instead of the other way around. Once the tool is in place however, another problem manifests itself. This often starts with a management mandate like, "OK n...

Now that we can test jcdTargets from a jcdUnitTester , the final step is to use a combination of JUnit and HTTPUnit to execute and verify the tests. Basically we turn jcdUnitTester into a simple RESTful Web Service where we post the tests. The first set of changes are to jcdUnitTester . My first pass at jcdUnitTester hard coded the test into the JCD. Since we'd like to change the message per test, it'd be better to pass this in as parameter on the HTTP Request. Use the getRequest().getParameterInfo().getWebParameterList() method of JCAPS' HttpServer class to get these values. Other values we'll need, in addition to the test itself, is the queue/topic name where the test will be placed and the queue/topic where jcdUnitTester will listen for the response. Use the same technique described in the previous post to set the replyTo field in the JMS message. This should be the minimum set of values we'll want to send to jcdUnitTester . Some other optional t...

Last time I explained how to trigger a JCAPS JCD from a web browser. This entry will hopefully clarify some of the items in that post, before building on that functionality to execute JUnit tests using HTTPUnit . First, the JCD created to listen for HTTP requests (let's call it jcdUnitTester ) will drop messages on a queue or topic where the JCD we want to test is listening (call it the jcdTarget ). The jcdTarget normally processes the incoming message, then passes the result to the next process via a queue, topic, or some other transport mechanism (call it targetResponse ). For jcdUnitTester to return the message to browser in the HTTP response, jcdUnitTester must be listening on targetResponse (creating a loop between jcdUnitTester and jcdTarget. ) In my code, most of my jcdTarget s involved JCDs sending responses back on a topic or queue. To make the JMS jcdTarget s dynamic, I've taken advantage of the JMS eGate class's sendTo method. The targetRespons...

Since starting with JCAPS last November, I've looked for a better, automated way to test my code. During JCAPS training, the test exercises are kicked off by placing files into a "hot folder" monitored by a File eWay. It's very cumbersome to monitor the directory after placing the file - constantly refreshing the window, wondering why the file is not being picked up - or what is taking so long. In my company's environment, the JCAPS development server ran on a shared machine that was cumbersome to access - adding to the difficulty in dropping this file. Since most of our components listened on JMS queues or topics, dropping messages directly to these queues would've been an option, but these were not accessible outside the machine. I knew there had to be a simpler, more elegant way to trigger our tests. What I needed was a way to drop a JMS message into the system and trigger my JCD. What I wanted was a servlet-like mechanism that I could trigger at will...

Things have been really hairy at work lately. I haven't had much chance to write. Here's what I've been up to. JCAPS One of my goals this year has been to make JCAPS more usable. A tall order indeed. I'm going to hit on the details of some of these items in later posts, but here's a bullet list of some of the issues I've been working through. My organization uses some non-standard x12 EDI messages. The way to get JCAPS to process these messages is by creating an OTD from an SEF file. I had no luck finding someone to help me create such a file and there are surprisingly few Java libraries that support these x12 messages. I wound up writing my own parser. It works pretty well and can be extended to support other message types. I've developed an approach that provides an automated way to test my JCAPS collaborations using JUnit. I'll describe this approach in future posts. The company I'm working for has a change control process th...

The JCAPS training I attended ( Foundations of Java CAPS II ) was excellent. We screamed through the course material in 3 days and had discussions on other JCAPS topics the other 2 days. Here were my impressions. I enjoyed learning about the eInsight Business Process Manager . The tools and steps involved in creating a business process are very similar to those used to create a JCD. There are a lot of interesting constructs like correlation and the "flow" element, which easily enable parallel processing. One frequent topic of discussion, was when to use an business process (eInsight) rather than a JCD. There's a certain flexibility to creating a business process, gained at the cost of speed. Since you can do virtually everything in a JCD that can be done by a business process, we questioned when each was a better fit. We didn't reach a consensus and I'm sure we'll be talking about this again. We explored the shortcomings of the repository and the be...

I'm back at JCAPS training this week. This time the focus is on the eInsight Business Process Manager . eInsight is the JCAPS architecture layer used to combine web services. The services are connected using BPEL which is an XML-based language to define business processes. The latest version of NetBeans also includes a BPEL tool, and I'm anxious to see how they compare. We have some time scheduled at the end of the formal training to explore some of the JCAPS problems we've been experiencing. In addition to the problems I've already outlined with the repository . I'm hoping we can also explore the following: Lately, we've seen some weird behavior where things that were previously working stopped. The errors in the logs don't seem to make any sense. At first we thought it might be due to some code that was copied to a different location, but I've seen this problem where no code was copied. The only remedy we've found is to recreate both the...

Making resolutions is tradition on New Year's Eve.... evaluate areas you'd like to improve (or new skills you'd like to acquire), and set goals to improve in those areas. Here are two work-related resolutions I'm going to focus on entering 2008. Make JCAPS Usable If you've been following my posts, you're aware that I've been frustrated with Sun's JCAPS product. The tool is simply not ready for prime time and it's hurting my productivity. In addition to the problems I've already outlined , developing in JCAPS' eDesigner makes me "feel dirty". You need to go outside to tool to create modules of shared code (jars). This limitation makes it hard to implement classes that follow even the simplest design patterns or integrate with third party frameworks, like Spring . I feel myself falling into bad coding habits. I'd like to find a compromise. What I'd like to do, is move my code out of the JCAPS JCD and into a sep...

I haven't had much chance to post since starting my new position. Getting up to speed in my new role has kept me busy. I have been using JCAPS though and thought I'd post a quick update on my impressions. Basically the good is still the good, but my team and I are experiencing more pain points, mostly due to the poor repository integrated into the tool. Here's a quick list of things I'd like to see improved. I've gotten in the habit of diff'ing my changes against the repository version to ensure I'm only checking in changes I've intended. JCAPS does not allow you to do this. There's also no easy way to diff changes between different repository versions. You need to be creative, and go outside to tool for this functionality. There's no easy way to find all the files you have checked out of the repository. You need to manually keep track of the files you've touched. I've become accustomed to relying on my development tools to ke...

I've been in JCAPS training all week, and so far my exposure to the product has produced mixed feelings .... the good.... JCAPS is definitely meant for enterprise-class application development. The architecture makes connecting to files, messaging queues, and databases trivial. It's scary easy. Things like monitoring and deploying to multiple environments are baked in. Again, it's nice to not have to worry about developing or searching for tools to manage these tasks. The JCAPS tools also produce some nice pseudo-documentation (diagrams and high level descriptions depicting what the application is doing and how things are connected). Some may think this is a little cheesy, but I like the idea of getting a high level picture of what is going on without having to surf through multiple source and configuration files. Having some built in transparency is nice (especially in large systems). the bad.... Is also the UGLY ... The JCAPS is built on NetBeans 3, whic...

In the midst of a job change, I've been spending most of the last few weeks handing my work off to others and writing some final documentation. With that almost complete, I'm turning my attention to my new position where I'll be working to integrate systems together in a Service Oriented Architecture (SOA). While I have SOA experience (creating & integrating web services as well as CORBA modules) my new company has chosen the Java Composite Application Suite (JCAPS) as it's SOA platform. It's (by far) the most comprehensive SOA solution I've had opportunity to work with. I'm really excited to get started. Here are some things I really like about the architecture and questions I have. The system is built on an application server and includes things like messaging, as well as consolidated user management and role based access built in. This is a big change from my current situation where I've needed to integrate (and implement) a lot of this f...