Open Source in the Data Integration Business

20 Nov 2009

I guess two of the huge growth areas are going to be Data Integration and Open Source, as companies are forced to consider these strategies, either due to technology change constraints or budget.

Obviously this begs a question, how are these two areas relevant to each other, is this a particularly strong growth area?

Data Integration tools are becoming indispensable in the Enterprise. As companies merge, acquire and grow, their systems need ways to share data. Cost can be a challenge, budgets do not always stretch to the IT change over, or if they do, it’s a planned migration sometimes over

years. Sometime technology just does not have an answer, one product, working well for a business with a user adoption because of the familiarity of the application, needs to be retained, and giving rise to the technology challenge of a data migration solution.

To help qualify what we mean by data integration, I recently read this definition: The process of combining data residing in different data sources and providing the user with a unified view of this data.

I guess, because of the economic constraints outlined above, Open Source is becoming a more compelling argument.

When we talk about Data Integration we traditionally mean Extract, Transform and Load, commonly referred to as the ETL process. This commonly suits two types of Projects, the Data Migration solutions, moving data from one system to another and the Data Sharing Projects, making the data available to be processed by other systems or as a view in other applications.

At OpenCRM ( we have been working on a number of projects that require these ETL processes, and the big question is how best to generic’ise and provide the solutions. Obviously, using Open Source tools to assist with the core migration or transformation tool helps all round. This keeps the cost down for the client and can (in some cases) speed up delivery, utilising tried and tested components, as a building block in the development cycle.

Are these building blocks useful for IT depts? I would suggest that yes, lots of IT depts that we work with are similar in structure to smaller Software Development companies, with their own development resource in-house and a management structure looking at the commercial relationship with finance depts. I would argue that these IT depts benefit enormously from Open Source code, partly because it allows them to focus on the end point of a project and not get too bogged down with the nitty gritty, which as we know can eat up valuable resources and take considerable time, letting them use the code as a building block, so that they can progress their deliverable’s quicker and (dare I say it) more visibly to their management team.

Should these IT Depts contribute something back to the Open Source communities that developed these building blocks, well yes frankly, that’s the whole point of Open Source, especially if they are going to be distributing the code to other parties within their group or sister companies.

So, Open Source Data Migration tools, a good thing to be involved with over the coming years? Yes, I think they are…. I know that OpenCRM has already embarked on a number of integration modules, using community led tools that have benefited from the commitment of resources from Software Add-ons.

If you’re interested in hearing more about Software Add-ons – The Home of OpenCRM, please click here, we would love to hear from you.