JCL

This is the home of DBIx-JCL, a Perl module that provides an application framework for developing database Extract, Transform, and Load (ETL) applications.

The JCL module can be used with Perl to write entire backend systems to maintain many different types of databases. The module has been used to write ETL applications that access Oracle databases, SQLServer databases, and Teradata databases. Any database that can be accessed by the Perl DBI module is supported. The list of DBI supported databases includes all the major database management systems.

ETL Tools

There are approximately twenty major players in the ETL Tools space according to the etltool.com survey. There are no doubt many more. The major offerings are very powerful. Such features as GUI interfaces for task design, connection to multiple DBMS platforms, the ability to use Meta Data for task design, the ability to run on many types of advanced hardware, and of course the promise of support and huge productivity gains. Be prepared to pay upwards of thousands of dollars per seat. Given this, choosing an ETL Tool is one of the most critical decisions you will make during the life cycle of your database.

JCL And ETL

JCL is an Open Source ETL Tool. The cost is free, but there is no paid support. If you are good with SQL, or you are already a Perl programmer, JCL may be a good fit for you. If you have Perl expertise in your organization, or you are looking for a tool that will allow you to expand your Perl skills, JCL is an excellent choice. The performance of JCL is excellent. JCL is built on top of the Perl DBI module. The Perl DBI module is the database access layer that JCL uses. The Perl DBI is written in C and has thousands of labor hours of tuning behind it. It is one of the fastest database access engines available and it supports access to a large number of DBMSs. JCL also takes advantage of native client tools such as SQLLoader on the Oracle platform for bulk loading of data.

Case Study

The system that I maintain is a medium sized Data Mart in a Foutune 100 company. The database is used to store HR data for all the the company’s internal web self-service HR applications. There are currently 200 JCL jobs writting for various load and extract operations. All of these jobs run on general purpose Unix hardware. Development of the JCL jobs is done on both Unix and Windows. The system has been in production for over two years.

More Information

If you have questions not answered on these pages, please feel free to contact the author. Contact information is available on the About page.