How to implement the splitter and aggregator patterns with Apache Camel

January 19th, 2014 by Ghaith Alrabadi No comments »

I have found that Apache Camel is a good way to load data from log files into a database. Read on to see how I did this using the splitter and aggregator patterns with Apache Camel.


» Read more: How to implement the splitter and aggregator patterns with Apache Camel

A Better Analogy For Agile Software Development?

December 30th, 2013 by Cecil Williams 1 comment »

For years, like many of you, I have been comparing software development to construction. But ever since adopting the agile methodology a decade ago, I have been looking for a better analogy to help me explain agile software development. I recently came up with what I think is that analogy – urban planning. Read on to see if you agree.


» Read more: A Better Analogy For Agile Software Development?

SphinxSearch | Full Text Searching

November 21st, 2013 by Stephen Dunn No comments »

This article explains how to use SphinxSE, Sphinx real time indexing, and set up Sphinx in order to improve search query performance.   First some background about issues surrounding full-text search.

Latest posts by Stephen Dunn (see all)


» Read more: SphinxSearch | Full Text Searching

Pizza & Beer: Learning with Atlassian

October 24th, 2013 by Amy Hughes No comments »

Atlassian Summit 2013 Logo

Atlassian Summit 2013 Logo

As a new employee here at Source Allies and a new user to our suite of Atlassian products, I was fortunate, and a little intimidated, to attend Atlassian’s 2013 Summit.  Despite my initial trepidation, I left feeling inspired by what I learned.

» Read more: Pizza & Beer: Learning with Atlassian

Pain Driven Learning

October 16th, 2013 by David Kessler No comments »
I find that in software development, and also in life, people learn best within the context of painful experiences.  I’m not suggesting that software development mentors go out of their way to create painful experiences for their teams.  On the contrary, just start listening.  It doesn’t take very long to identify pain points within a team.  Equipping teams to solve their own problems is drastically more effective than teaching people the right way to think.   When I teach large groups I often describe my points as hooks or link where they can go back and expand their understanding.  This large-scale map is essential so teams have a shared road map.  But this is rarely enough.
Understanding is built incrementally.  Each hook requires layers of experience and reflection hung upon it to develop a robust context of understanding.  While there are a vast number of reasons that people are motivated to learn, as a mentor I find pain to be a constant source of effective mentoring.  Pain is easy to identify within a group if you take the time to listen.  As a mentor I search for the root causes that set this frustration in motion.  Once I’ve collected and researched a variety of pain points I look for common themes and related root causes.
I remember one team that had a significant trust issue between the business team and the development team.  The most obvious place that this pain point appeared was in the daily stand-up meeting.  The business members were quiet and withdrawn while the development team dominated the conversation.  As I started asking questions I quickly realized that there was a language barrier within the team.  While most everyone on the team grew up in the same country, the developers consistently used technical terms that confused and alienated the non-technical team members.  On top of this divide the technical team regularly used their exclusive knowledge to push decisions through.  After discussing this with the whole team we adopted a “no tech talk” policy at stand-up.  In fact we gave non-technical members cards that they could throw down in the center of the group if the speaker started using terminology that only part of the team understood.
This small but powerful modification significantly changed this team.  The business team stopped reading their emails and texts during stand-up and started engaging in meaningful conversations.  The technical team started translating their technical thoughts into business terms.  Changing how the team communicated not only aligned their conversation with the business, it began to leak into their code.  Constantly thinking about their work in the context of the business helped them create code that reflected the business.  Clearer code reduce unnecessary overhead related to translating business terminology into application terminology.  This increase clarity also cut down wasteful communication overhead and reduced the new team member learning curve.  Aligning this team was not just a feel good moment; it resulted in a critical alignment of their software with the business.
While this example focuses on “soft skills”, pain driven learning is applicable to every part of software development, technical or not.  I focus on pain points because they’re easy to find if you just listen.  The emotion surrounding these problems provides an implicit invitation for a solution.  The act of helping a team solve a problem that they face increases trust and moral.  Once a team believes that they can solve the problems they face, they become emboldened to look at their challenges and take steps to correct them.  These shared experiences draw the team together and provide a powerful environment for developing valuable software.  Mentors earn respect from their teams by helping them identify and overcome issues.  Respect opens avenues for powerful teaching opportunities that extend beyond pain points.
Effective mentoring is not delivered by force.  On the contrary, effective teachers must become students willing to listen and learn.  In the same way that effective software provides value to the customer, effective teaching equips students to keep moving forward.  The correct posture of an effective mentor is humility.
I find that in software development, and also in life, people learn best within the context of painful experiences.  I’m not suggesting that software development mentors go out of their way to create painful experiences for their teams.  On the contrary, just start listening.  It doesn’t take very long to identify pain points within a team.  Equipping teams to solve their own problems is drastically more effective than teaching people the right way to think.   When I teach large groups I often describe my points as hooks or link where they can go back and expand their understanding.  This large-scale map is essential so teams have a shared road map.  But this is rarely enough.

Agile Iowa No Estimates Puzzle Experiment

October 2nd, 2013 by Cecil Williams No comments »
cecil-presenting-agile-iowa-puzzleI facilitated my own rendition of the #NoEstimates Puzzle Experiment for the September 2013 Agile Iowa user group meeting. This experiment was created by Chris Chapman to generate critical thinking and conversation concerning whether estimates are necessary to produce quality software. The meeting had a great turnout, with around 40 people attending during a Midwest thunderstorm that left over 20,000 people without power.

» Read more: Agile Iowa No Estimates Puzzle Experiment

Getting Started with Camel: Error Handling

September 16th, 2013 by Ben Kiefer No comments »

Error handling is tricky. Not because it’s especially hard to do, but because everyone (operations, the business team, fellow programmers) seems to have a different idea of how a particular situation should be handled. A web service is down? No problem. You should try again every five seconds, but no more than 10 times. If the service doesn’t respond, send Operations an email, but don’t send me an email every time you fail to message it, just the 10th time.

These special requests result in “little gems” of code that are sprinkled throughout your application. They’re really important when everything is going wrong and ignored the rest of the time. It’s a shame really, some of the ridiculous stuff above is harder to write (and test) than some of the production code we’ve all written.

I know I’ve said this before, but I like Camel. It makes all the silly requests above trivial, and it gives me a mechanism for testing that I wired everything up correctly. Let’s look at a few ways to deal with errors that occur in your Camel routes.

» Read more: Getting Started with Camel: Error Handling

Pragmatic Application of Principles

September 13th, 2013 by David Kessler No comments »

I was reminded of a profound truth as I was re-reading Robert C. Martin’s book “Agile Software Development, Principles, Patterns, and Practices”, in C# this time.

It is not wise to apply (a) principle … if there is no symptom.

» Read more: Pragmatic Application of Principles

What you missed at Agile 2013

August 26th, 2013 by Cecil Williams 1 comment »

I was able to attend the Agile 2013 conference in Nashville, TN earlier this month. I had previously attended Agile 2006 in Minneapolis, MN. There was a significant difference.

At this year’s conference, the overall theme seemed to be that teams needed to focus on producing value rather than following a process.  Most of the attendees have figured out that the process will only get you so far.  At that point you have to figure out how to improve what you are delivering, not what you are doing.

» Read more: What you missed at Agile 2013

Agile Manifesto – Responding to Change Over Following a Plan

August 19th, 2013 by Cecil Williams No comments »
Is it really possible that intense planning and the ability to respond to change can co-exist within the same development process? If you are wondering this, then you are not alone. Clients regularly ask us if Agile software development teams follow any sort of plan or are they just feel good, free for alls? In this article we explain the types of processes that can be adopted to allow your teams to plan while still responding to change.
Traditional Software Development
Traditionally, software development teams viewed the cost to change something as increasing over time.  To save money these teams required customers to define everything they wanted before they ever started building the system.  While this sounds reasonable, we all know that something will always be missed.  These missing requirements will creep into the project little-by-little.
Many organizations try to prevent changes by making it very difficult, if not impossible, to inject changes after the initial planning phase.  This resistance drives many teams to add everything, including the proverbial kitchen sink, to their initial requirements to avoid the pain of adding changes later.
While this seemed like a good idea at the time, our industry has learned that allowing change is imperative for companies that must compete in a fast paced, cut throat, rapidly changing marketplace.
The last decade has seen several software development processes become popular that allow for planning and change to happen at the same time.
Agile Software Development
Agile software development processes accomplish this through different levels of frequent planning and re-evaluation. The first and highest level of planning is release planning.  Within release planning, teams agree on the features to include in a release.  At this level of planning, features remain very large.  The purpose of this planning session is to paint a large picture of the project with very large paint strokes.  Once these large strokes are grouped into meaningful releases, we proceed to prioritize each release and then break them down into smaller time periods known as iterations, based on priority order.  Each iteration is typically limited to two week periods.
At the beginning of each iteration we agree on which features the team thinks they can complete.  These features are broken down into stories, which are kept in a prioritized feature list.  By working in iterations, the team has the ability to adjust what they are working on every two weeks.  Thus, if a new requirement is discovered, the team is able to incorporate it without difficulty in the next iteration.
This approach intensifies the investment of planning as the team gets closer to a given iteration. Focusing on a few features intensely empowers us to lock in a limited number of decisions that often uncover information that effects later decisions.  While the old style of up-front planning attempts to uncover these pieces of information early, a significant amount of this information cannot be fully understood until we start to develop and use a working system.  No amount of thinking can replace hands-on experience with the real system.
As the team works in an iteration, the stories are pushed from one stage to the next. The stages vary from company to company, but typically include development, testing, and deployment.
Once an iteration is completed we take time to reflect on what we learned.  This review meeting is called a retrospective.  In this meeting the team discusses what went well, what did not go well, and what changes we want to make.  This meeting helps us learn from our mistakes and make a commitment to improve.
Lean Software Development
Taking the Agile software development process a step further, the lean software development approach focuses on limiting the work in progress based on the constraints of the team. The team cannot push work to the next step.  Instead, the downstream team members pull work into their stage when they are ready to work.
This is done to prevent any one stage in the process from completing more work than the other stages can process. If one stage finishes work and the next stage cannot work on it, then this is considered wasted inventory. Lean software development borrows the theory of constraints from lean manufacturing, in that it limits the work in progress to the slowest stage in the process.
By limiting the work in progress, teams are able to identify bottlenecks in their process.  These bottlenecks may be due to resources, bureaucracy, time, distance, or any number of reasons.  Lean software development teams focus on eliminating their bottlenecks so that they can function at their optimum.
Lean software development processes do not use fixed time periods like iterations.  Instead, a lean process will continue to pull work at the rate that it is capable of producing features.  And rather than having a scheduled release, the team will simply release features whenever the team determines is appropriate.
Lean Startup
Teams are starting to go even further than the lean process, by combining the lean process with ideas from startup companies.  These teams are focusing on a single feature and all the variations that a user might do with that feature.  The lean startup process uses value stream mapping to determine what are the most important parts of a feature to the users.  Then they build those parts of the feature and deliver it to the users.  This allows them to get even quicker feedback and validate whether they are building a desirable product.
Final Thoughts
The last decade has seen several software development processes emerge for responding to change over following a plan without creating destructive chaos.  Following an intensely focused plan is far superior to creating a bloated, expensive, “perfect” plan up-front that cannot be changed without adding significant cost.  Responding to change is not only more effective, it is imperative for companies that must compete in a fast paced, cut throat, rapidly changing marketplace.

By David Kessler and Cecil Williams

Is it really possible that intense planning and the ability to respond to change can co-exist within the same development process? If you are wondering this, then you are not alone. Clients regularly ask us if Agile software development teams follow any sort of plan or are they just feel good, free for alls? In this article we explain the types of processes that can be adopted to allow your teams to plan while still responding to change.

» Read more: Agile Manifesto – Responding to Change Over Following a Plan