MFT Gateway Architecture Guide
This guide provides a detailed view of the architecture of the MFT Gateway platform.
1 Overview
The MFT Gateway is a cloud native ‘Serverless’ application built on top of Amazon Web Services (AWS). It allows the exchange of files between partners, utilizing the secure Applicability Statement 2 (AS2) protocol. The key AWS services used are as follows:
| AWS Service | Utilization |
|---|---|
| S3 | Storage of files exchanged and MDNs etc |
| DynamoDB | Metadata database. Storage of message details, timestamps, partner details etc |
| Cognito | Authentication and user management |
| Lambda | Processing, including AS2 encryption/decryption, signatures etc |
| API Gateway | Incoming AS2 message acceptance, REST API endpoint hosting etc |
| SNS | Internal signalling |
| EC2 | Large payload handling (where message sizes exceed API Gateway and Lambda thresholds; see this section regarding supported sizes) |
| Timestream | Time series data storage and statistics |
| Transfer | SFTP access |
| VPC | Static IPs for outgoing messages |
2 Deployment Architecture
The MFT Gateway exposes two endpoints for accepting messages over the Internet. Namely, the (default) API Gateway based endpoints, and the EC2 based Large payload endpoints. While the API Gateway based endpoints are the preferred option for accepting messages, they have an inherent limitation on the maximum size of messages which can be processed. If the messages expected from a partner are generally small messages not exceeding 5MB in size, the API Gateway based endpoints offer the most scalable option. However, if larger messages are expected, the Large payload endpoints exposed by the auto-scaling EC2 servers are the right option. These EC2 servers only perform IO, and reads requests received over the network, and persists them into S3. Thus, small instances without high CPU or memory resources are ideal.
Actual message processing, such as AS2 message validation, decryption and signature verification happens via Lambda functions. Message meta-data is persisted to DynamoDB, while the decrypted attachments, and the generated MDNs are saved into S3. SNS based internal signalling performs Email and Webhook notifications.
The S3 bucket exposes the received attachments for native S3 integration with external systems. In addition, Transfer allows SFTP access to the S3 bucket. The REST APIs exposed by the MFT Gateway accept requests through the API Gateway, and serves these requests through Lambda functions which access DynamoDB and S3.
The web based user interface is driven by static HTML pages hosted on S3 with cloudfront, coupled with API Gateway and Lambda based APIs, and Cognito based authentication and user management. This allows users to download received files, or submit new messages for transmission. When messages are submitted through the web console, S3 or SFTP integration, it is placed into the S3 bucket. This triggers the sending process driven by Lambda and Step functions, which will perform the AS2 digital signature and encryption, and actual transmission, with retry handling.
3 AS2 Gateway vs MFT Gateway
MFT Gateway is the recommended AS2 solution for both new SaaS customers, and large Enterprise customers. However, the MFT Gateway solution cannot be deployed on-premise, on virtual machines, Kubernetes, Containers or non-AWS cloud platforms. The AS2 Gateway offers a low cost and easy to deploy AS2 solution for such use cases. Here is a comparison between the two options.
| Description | AS2 Gateway | MFT Gateway |
|---|---|---|
| Available as Software as a Service (SaaS) with nothing to install | Yes | Yes |
| Available for on-premise deployment on VMs, Kubernetes or containers | Yes | No |
| Available for deployment on customer’s GCP, Azure etc cloud account | Yes | No |
| Available for deployment on customer’s AWS account | Yes | Yes |
| Dedicated hosting on customer’s preferred geographic region | Yes | Yes |
| Integration via SFTP & REST API | Yes | Yes |
| Integration via AWS S3 native access | Incoming only | Yes |
| Traditional ‘Server’ based using integrated DB (MySQL, PostgreSQL) | Yes | No |
| Highly scalable cloud native ‘Serverless’ solution with no SPF | No | Yes |
| Ability to process millions of large messages per day | No | Yes |
3 Installation
For SaaS accounts, there is no installation necessary, but a simple account registration. For dedicated hosting or enterprise accounts, the installation will be performed by the Aayu team at no additional cost. When deployment into an AWS account of the customer is required, the customer would first create a new AWS account under the AWS Organizations, and grant credentials to the Aayu team. Once Aayu team performs the installation and initial quality assurance testing, the deployment will be handed over to the customer for acceptance testing. Aayu team will assist the customer in setting up Stations, Partners, and any network level access configuration to establish necessary connections with the Partners and to successfully complete the testing phase. Customer can choose to revoke access to the Aayu team thereafter.
4 Scalability testing
The MFT Gateway has been lab tested at over 100 TPS with a 1MB file size, and observed to scale linearly. Since all files, MDNs and headers etc are stored on S3, the size of messages, or large accumulated message volumes, have no impact on the performance or stability of the system. The Lambda based processing allows the system to scale up and down based on the load handled, and where the EC2 instances are used, they too scale automatically without any single point of failure. Stress testing of the MFT Gateway will actually result in a stress test of its underlying AWS cloud and services, and will not show any visible limit of capacity.
Refer the article on AS2 Load testing for more information
5 Limits
File/payload size
- Up to 3.5 GB per outgoing message
- Up to 2.5 GB per incoming message