05 Oct 2022 Informatica Intelligent Cloud Services (IICS) – Part 1: Architecture and Services Overview

Welcome to the first article in the Informatica Intelligent Cloud Services (IICS) series, where we are going to run through its underlying architecture and the main services this tool offers. Informatica is one of the most important data integration vendors on the market; and the maker of PowerCenter, a well-known ETL tool, amongst other products.

The continued growth in the use of cloud applications and the complexity of effectively sharing and using business data across an organisation can lead to a lack of integration between these applications and local ones, making an effectively tested ad hoc integration solution such as Informatica Cloud a necessity for your integration needs.

Why Informatica Intelligent Cloud Service?

Informatica Cloud offers iPaaS on the cloud that enables the integration of disparate data by updating and synchronising specific business processes and integrating data from different environments. Whether acting between cloud applications and SaaS platforms, or interfacing with application archives and databases on premise, Informatica Cloud provides the tools needed to design a custom solution.

Currently, IICS is ranked as a world leader for Enterprise Integration Platforms as a Service according to the Gartner Magic Quadrant, offering benefits such as:

• Increased agility: Process data integration tasks anywhere to keep up with market changes and evolving business requirements.
• Increased productivity: Build and run advanced integrations at scale, with pre-built templates and reusable mappings, without coding!
• High performance: Deploy and manage complex, modern workloads with enterprise-grade performance and reliability.
• Simplified integration: Process complex data integration mapping tasks with a serverless Spark compute engine.
• Cost optimisation: Select the most cost-efficient processing option based on usage patterns and behaviour.

Informatica Cloud is also an ETL tool and offers both batch and real-time processing.

If you would like to get more hands-on with IICS, a 30-day trial can be requested here.

Informatica Cloud Architecture

Informatica Cloud architecture has two main components, Informatica Cloud and the Secure Agent:

Informatica Cloud

This is a bundle of hardware and software provided by Informatica; we can utilise the resources of the cloud and run applications on it. Unlike PowerCenter, there is no need to install any client applications on your PC, as you directly access the applications through your browser.

You can find more details in the Informatica Intelligent Cloud Services section.

Secure Agent

This is lightweight software installed on a server behind the organisation firewall, providing connectivity between the cloud and the on-premise system. The Secure Agent processes the data locally and securely; it also acts as a medium to transfer data between the cloud and the organisation’s on-prem setup.

Informatica Cloud automatically updates the Secure Agent software and restarts it – organisations can focus on developing applications and do not need to worry about administration. The Secure Agent runs three major services on the server, listed below:

• Data Integration Service: This runs batch jobs, executing data integration jobs such as mapping, task, and task flow instances.
• Process Integration Service: This runs real-time APIs, executing application integration processes, connectors, and connections.
• Common Integration Components: This executes shell scripts or batch commands in a Command Task step of a task flow.

Informatica Intelligent Cloud Services (IICS)

IICS provides various services that can used to manage the application, and for developing and monitoring jobs. The screenshot below shows the services provided by Informatica, depending on your licence:

Administrator: Where you can download the Secure Agent and install it on your server, as well as connectors to applications like files, databases, and so on.

Data Integration:Used for developing batch jobs. The different tasks used for developing batch jobs are listed below:

• Mapping Task: Like PowerCenter mapping designer, this is used to define the data flow logic to process the data.
• Synchronization Task: This is used to synchronise data between the source and target. It provides DML operations like insert, update, upset and delete.
• Replication Task: This can be used to replicate more than one object from a source to a target. You can either use full load or incremental load options to replicate the data.
• Taskflows: You can combine other tasks and run them together in a flow, in sequence, in parallel, or based on a decision.

Monitor: as in PowerCenter, you can monitor the batch jobs developed in the data integration. You can check running jobs, failed jobs, and successful jobs. It also provides options to restart a job, stop a job, and download logs.

Application Integration: This is to develop Informatica Cloud real-time (ICRT) APIs. The different types of assets that you can create are:

• Process Object: This defines the structure of the request and the response of the API; the structure of the temporary variables can also be defined here.
• Service Connectors: A service connector defines what actions can be performed on an application. Before connecting to a third-party application, a service connector should be defined.
• App Connection: An app connection should be defined on top of the connector. The app connection contains the credentials that the connector uses to perform the action with third-party applications.
• Process: To develop an API. This contains the business logic that the API processes. Informatica Cloud automatically generates the REST and SOAP APIs when you develop and publish a process.
• Guides: Interactive screens can be developed using the guides where the user can input and view the data.

Application Integration Console: You can monitor real-time processes and schedule them; you can download logs and monitor server health.

API Manager: You can set policies on the API; policies include the IP addresses from where the API can be accessed and the number of API calls per minute (Rate Limit). You can also define the authentication mechanism for each API using the API registry. The API group provides an option to group logically related APIs into a single group, and the policies can be applied to these groups.

Discovery IQ: This provides analytics on the number of jobs and the processes run in your organisation.  We can get some key performance indicators like the number of successful jobs, failed jobs, data volume, etc. We can also download logs, analyse them, and generate some metrics from them.

Operational Insights: This gives the status of Secure Agent services and whether they are up and running or not.

Informatica Cloud Connectors

When we want to connect to an application (whether it is a file system, a database, or a cloud application) we need a connector. Informatica Cloud has a lot of built-in connectors available for this purpose, but you need to have the licence for them. Some of the most used are File, FTP, JDBC, Salesforce, Snowflake, Amazon AWS, Microsoft Azure, Hadoop, Oracle, and SAP.

There is no need to install any client applications. All the apps can be accessed from the browser, where we can develop the task flows, guaranteeing faster development and no loss of code if there’s a network failure.

Identity and Access Management

IICS supports a variety of authentication mechanisms – password-based, SSO-based, certificate-based, and token-based. Multi-factor authentication mechanisms like trusted IP address ranges also enable stringent security.

For native password-based authentication, user credentials are hashed and securely stored in the IICS host. Administrators of the customer organisation can configure policies for password strength and rotation to suit their business needs.

Administrators can also assign different roles to users to maintain the principle of least privilege: users are only granted access to the capabilities needed to perform their functions, allowing control over who does what, so that some users are managing project and folder structure, some are designing, some are running jobs, and so on. IICS provides predefined roles to facilitate user role assignment for the most common needs, and custom roles for administrators to define new roles to meet the specific needs of their organisation.

Cost Optimisation

When evaluating a cloud-native data management platform, enterprises must take a close look at its data processing capabilities to maximise ROI, optimise usage, and control costs.

As a next-generation intelligent Data Management Engine, IICS delivers radical simplicity, improves user productivity, and provides elastic scalability.

IICS provides many features that help to reduce costs, such as Smart Scheduling, Smart Cluster Management, and Storage Auto Scaling. As an example, let’s see how IICS optimises costs with one of them:

In terms of job execution, lack of storage is worse than lack of nodes because the former will result in job failure while the latter only causes slow performance. A lot of user intervention is required, usually for an on-premise cluster. When a job fails with the disk error, the user will have to manually reconfigure disks for each node and then restart those impacted jobs. Besides, storage cost can be high: one t2.2xlarge with 1TB general-purpose SSDs (EBS-backed) can cost around $1.5K per month. Storage Auto Scaler constantly monitors the disk usage of each node and automatically attaches additional storage volumes only to those nodes that require more storage.

The Informatica Data Management Engine eliminates user intervention entirely and dramatically reduces storage cost.

Conclusion

This concludes the first article in this series about Informatica Intelligent Cloud Services (IICS), in which we have given an overview of its architecture and the services it offers, making it an easy-to-use tool, whether for the development of new features or for access management.

In our next blog post we will look at a specific use case to demonstrate how easy it can be to work with IICS, or more precisely with its data integration functionalities with the collaboration of the Snowflake Data Cloud, and how we can parse semi-structured data in JSON. Stay tuned!

In the meantime, if you’d like to find out more about how these technologies and tools can help your business, don’t hesitate to get in touch with our global team of certified consultants!