Wednesday, April 17, 2013

Delimited text processing using camel bindy

Processing a delimited text is one of the most common scenarios that developers come across. From parsing a line of text to processing huge csv files as a batch job or an etl job.
As a programmer, i always wondered if there was a libray that just converted the delimited text to pojos so i could just define and work on my pojos. And yes, there is, camel-bindy.

Apache camel is an open source integration framework that supports numerous enterprise integration patterns. Even though apache camel has numerous components that supports different integration points, one need not worry about all of it to use camel-bindy. Camel bindy is the apache camel component that supports processing of delimited text and binding it to POJOs.




This writeup assumes that the reader has a basic knowledge of camel. Dont worry, if you are new to apache camel, a few minutes of reading apache camel will get you started.



Lets look at how camel-bindy can help to bind the comma delimited text to Employee bean. The text could be a record form a large csv

file. Here the writeup focuses more on how to deserialize each record from a large file rather than the file processing itself.



Consider the following text that need to be deserialized to an employee object for further processing.
"john, doe, d1, true, city1, state1, 1234"

Below is our Employee class.

























and the Address class is,



























camel-spring, camel-core and camel-bindy jars are required to build and run this example.
If using maven, the following dependencies are required.





















Route is the core of camel. We will define the route as a java dsl. A route is defined in a RouteBuilder class as below:



























The spring configuration for this example looks like this:

classpath:camel-spring-context.xml






Now, lets execute this. We will use a java main method as below:
















The following is the output:

Employee Name is john doe.

Thats it. We have successfully converted a comma delimited text to a java object. We just used a small text. Imagine how helpful camel bindy is when you want to process a record with many more delimited fields.

If you wanted to process all csv files in a folder, the route would just be:

from("file://xyz")
  .unmarshal(bindy)
  .split(body())
  .process(....);

This is a very basic example to show how the csv to pojo binding can be achieved using apache camel.

Hope the post was useful. Please leave your comments.

My other posts:

http://techusergit.blogspot.com