The ETL (Extract-Transform-Load) are one of the most critical
processes of BI and Data Warehouse applications.
What is ETL?
The Extract-Transform and Load process consists of the following three sub-processes
which are used to transfer data from production systems to the data warehouse where
they are consumed by BI applications:
of the data from production application and databases.
the data to reconcile it across source systems, including required data cleaning.
Also the data
is transformed to meet the requirements of the target systems(StarSchema,Slowly
of the transformed data into the Data Warehouse, Data Marts and other BI applications.
Enterprise ETL Vendors:
DataStage and Informatica are examples of Enterprise ETL solutions
that have extensive background and experience in the ETL market with their ability
to scale performance in handling very large data volumes in complex, heterogeneous
environments. These products provide comprehensive features and functionality and
so require extensive training to use effectively.
How Competent Systems helps:
Competent Systems goal is to help customers optimize the value of all their information
assets. We deliver the high-quality, low-risk information management solutions needed
to boost business results. Competemt System's solutions are designed to,
sales revenues and shareholder value.
pre-implementation ROI projections with post implementation ROI results ensuring
that our solutions
deliver as promised.
The Informatica solution for enterprise data warehousing is proven
to help IT departments implement data marts and departmental data warehouses and
readily scale them up to enterprise data warehousing environments. This solution
serves as the foundation for all data warehousing and enterprise data warehousing
projects. It accelerates their deployment, minimizing costs and risks, by ensuring
that enterprise data warehouses are populated and maintained with trustworthy, actionable,
and authoritative data.
A Proven Solution for Enterprise Data Warehousing:
The Informatica solution for enterprise data warehousing empowers the IT team to
easily adapt to changing business requirements. With the Informatica solution for
enterprise data warehousing, your IT organization can provide your business with
the data it needs for:
• More comprehensive
visibility across business units.
transparency for regulatory compliance.
ETL (Extract, Transfer and Load) Accelerator:
The ETL architecture will give BI and other projects a jump start in analyzing,
designing and building ETL interfaces. Informatica provides specific insight for
common ETL programming patterns such as common error handling mappings and application
configuration guidance. The designs accompanying the code samples should be considered
vendor-independent and can be leveraged in ETL projects, whether hand-coded or built
from other vendor tool suites.
End-to-End Data Migration:
Most projects design and build conversion architectures using customized code, which
increases the costs for Competent System's and its clients because the development
process is slow and requires a high number of skilled resources. Competent System's
Solution Works provides end-to-end conversion services by utilizing Informatica's
ETL software to perform conversion development instead of building customized code.
This conversion service offering saves at least 30 percent in conversion costs and
reduces conversion development time by several months.
DataStage integrates data on demand across many systems via a
high performance parallel framework, extended metadata management, and enterprise
It integrates data on demand with a high performance parallel
framework, extended metadata management, and enterprise connectivity.
the collection, integration and transformation of large volumes of data, with data
from simple to highly complex.
• New features,
such as an operations console, an interactive debugger for parallel jobs, and support
optimization, help customers work smarter, enhance productivity and accelerate problem
a scalable platform that enables companies to solve large-scale business problems
processing of massive data volumes.
real-time data integration.
developers to maximize speed, flexibility and effectiveness in building, deploying,
updating and managing
their data integration infrastructure.
connectivity between any data source and any application.
Features and Benefits:
The powerful ETL solution supports the collection,
integration and transformation of large volumes of data, with data structures ranging
from simple to highly complex. IBM DataStage manages data arriving in real-time
as well as data received on a periodic or scheduled basis.
The scalable platform enables companies to solve
large-scale business problems through high-performance processing of massive data
volumes. By leveraging the parallel processing capabilities of multiprocessor hardware
platforms, IBM InfoSphere DataStage Enterprise Edition can scale to satisfy the
demands of ever-growing data volumes, stringent real-time requirements, and ever
shrinking batch windows.
Comprehensive source and target support for a
virtually unlimited number of heterogeneous data sources and targets in a single
job includes text files; complex data structures in XML; ERP systems such as SAP
and PeopleSoft; almost any database (including partitioned databases); web services;
and business intelligence tools like SAS.
Real-time data integration support operates in
real-time. It captures messages from Message Oriented Middleware (MOM) queues using
JMS or WebSphere MQ adapters to seamlessly combine data into conforming operational
and historical analysis perspectives. IBM InfoSphere Information Services Director
provides a service-oriented architecture (SOA) for publishing data integration logic
as shared services that can be reused across the enterprise. These services are
capable of simultaneously supporting high-speed, high reliability requirements of
transactional processing and the high volume bulk data requirements of batch processing.
Advanced maintenance and development enables developers
to maximize speed, flexibility and effectiveness in building, deploying, updating
and managing their data integration infrastructure. Full data integration reduces
the development and maintenance cycle for data integration projects by simplifying
administration and maximizing development resources.
Complete connectivity between any data source
and any application ensures that the most relevant, complete and accurate data is
integrated and used by the most popular enterprise application software brands,
including SAP, Siebel, Oracle, and PeopleSoft.