ETL Agent - Azuresupport Templates, Services with Bicep und Terraform

Business Challenge

In today's business operations, the transfer of files between multiple environments (Linux and windows), applications, databases, and folders is crucial. However, IT departments often rely on manual solutions such as scripts and programs to transfer data. This approach can lead to potential SLA and security breaches, and may adversely affect the quality of services.

The ETL-Agent provides a solution for managing your business data and facilitating file transfers between environments.

Try it now > Go to case study >
(Get Your First Process for Free with Our Self-Service in 2 Minutes.)

Our solution provides central monitoring for complete documentation and traceability of all processes. It includes a console app based on .NET Core with a workflow engine that runs on various operating systems and environments. Additional modules can be added to extend the app's functionality.

The ETL-Agent can be controlled both locally and centrally and runs on various operating systems such as Windows and Linux, as well as in environments such as mainframes, microcontrollers and smart devices.

We offer the ETL-Agent Managed File Transfer (MFT) to minimize risks and expedite the delivery of business services. It allows you to design file transfer workflows within our system for seamless integration and automation.

Our solution encrypts configuration options such as credentials and connection strings, which can be stored in KeyVault for enhanced security.

Connect | Integrate | Automate

Manage your processes as a consultant

Listing of processes.

The consultant can manage agent processes and assign multiple flows to the process.

A BPMN flow editor

The consultant can manage flow data and create BPMN diagrams for clarity.

Export into word

The consultant can export the whole process into a word document.

Process steps

Design the File Transfer Process

Describe the current and target situation, then visualize it with BPMN

Realize your process

Configure the Agent on a VM to realize your process.

Transfer and archive your files

The ETL transfers files between any system and can keep archives of files for backup.

Realtime process overview in ETL-Web

Monitor growth, replication, and backups and decide performance updates, upgrades and maintenance from the log information in the Etl-Web

Managed File Transfer

File transfer: The ETL agent can be configured for file transfer and monitoring via a unified interface, with automatic configuration saving and execution upon completion.
Transfer files securely: We support secure file transfer and copying between various systems, including SAP, SQL, SFTP, Google Cloud, and more. For example, our ETL agent facilitates the transfer of personnel bookings using md5 encryption via SFTP to SAP on the OS level (Oracle Unix) and pushes the bookings via RFC in SAP for increased efficiency and security.
Central Configuration: Configuration of the agent is limited to authorized administrators.
High Availability: The application runs on a highly available Azure App Service, ensuring scalability and reliability.

Frequently Asked Questions

Our FAQ section provides answers to common questions about our system and its capabilities. Get information about security measures, authentication methods, data transfer, file filtering, supported OS and systems, on-premises deployment, and more.

1Is your system available for on-premises environments?

Our system can be deployed on an on-premises environment.

2Is it secure and easy to transfer data between Windows and Linux using the agent?

Yes, the agent provides a secure and easy way to transfer data between Windows and Linux operating systems.

3Can the agent transfer data between different types of devices?

Yes, the agent can transfer data between a range of devices, including computers, servers, mobile devices, and other devices that operate on supported operating systems like Windows and Linux.

4What type of data is transferred by your system?

This system is capable of transferring any type of file, and it also offers the ability to filter files based on specific patterns, such as "." (all files), ".pdf" (PDF files), ".csv" (CSV files), and so on.

5What security measures does your system employ?

Our system employs various security measures, including credentials and connection string encoding, SFTP for data transfer, HTTPS for communication, and more.

6What authentication methods do you use to access SFTP endpoints?

We offer several authentication methods, including username and password, password file, and ppk file with passphrase. Additionally, we have a tool that encode all credentials for added security.

7What authentication methods do you use to access SQL endpoints?

We support connection string for the agent to connect to the SQL server, and we also provide a tool to encode all credentials for enhanced security.

8How do you transfer files larger than 10 GB in size?

We utilize compression techniques to reduce the file size before transfer. The compressed file is then sent to the destination where it is decompressed. This method allows us to efficiently transfer files larger than 10 GB. For instance, we successfully transfer several files that are larger than 20 GB in size using this approach.

9What operating systems and systems are supported by your agent?

Our agent supports both Windows and Linux operating systems, and it integrates with various systems such as SFTP, Azure Storage, Google Cloud Storage, SQL, Email, SAP, and more.

10Does your system support high availability and data backup?

Yes, our system is highly available with robust failover mechanisms in place to ensure uninterrupted service. We also support data backup as archive on all systems, including SFTP, Azure Storage, Google Cloud Storage, and more, to ensure the safety and availability of your data.

11How much does the system infrastructure cost?

Our system's infrastructure cost is significantly lower than other systems, often less than 20%.

12How can we monitor the status of data transfer in the system?

We offer an analytics module which displays the real-time status of all running processes. You can view details about each file transfer, including the file name, size, location, transfer status, , permissions and more. The information can also be exported as a CSV file for further analysis.

13How do we document the process and does your system provide any features for it?

Our system provides a comprehensive documentation feature for the ETL processes. It includes a module for designing, creating, and maintaining details of each process. One of the sections provided is BPMN flow where consultants can design and store the BPMN flow for the process for future reference. Moreover, consultants can export the entire process into a Word document. This module will help you document the processes that the ETL agent is running effectively.

14Can a department monitor their own processes using the system?

Yes, our system allows administrators to assign relevant processes to departments, enabling them to have specific oversight.