Over the past few decades, the interpretation of a large file has drastically changed. Technology has evolved from handling a few kilobytes to managing tens of thousands of terabytes daily over the internet. The definition of a large file varies from system to system. While some systems can only handle megabyte-sized files, others are equipped to work with files as large as several terabytes. For instance, Gmail has a file size limit of 25 megabytes, but a system collecting images from the Hubble or JWST needs to manage files in the terabyte or even petabyte range.
Most AS2 systems are capable of handling file sizes in the 1-2 GB range by default. However, with proper configuration and system tuning, AS2 systems can manage very large files, exceeding 100 GB.
Like any other software, the file sizes an AS2 system can handle highly depend on the software configurations, such as memory limitations, resource constraints, and timeout settings. Even with adequate resources, how the system is developed to handle large files plays a role in how well the AS2 software behaves when signing or encrypting large files.
When working with very large files, the system running the AS2 software must be capable of handling the large files. There should be sufficient disk space, RAM, and temporary space to handle signing and encryption if needed.
The AS2 protocol transfers files over HTTP/S. While there are no strict limitations on the file size that can be transferred over HTTP, several factors determine the maximum file size in practice:
AS2 systems like AS2 Gateway or MFT Gateway allow users to compress payloads before transmitting them to other systems, significantly reducing the file size. This is especially useful for text-heavy files like XML or EDI. In some cases, compression can reduce the file size by 50% or more, using less network bandwidth and enabling faster transmissions.
The receiving system must be capable of decompressing the original file to complete a successful transmission. Nearly all modern AS2 systems support compression and decompression of data. This feature is invaluable, allowing AS2 users to send larger files without requiring more memory or network bandwidth.
Since the AS2 protocol transfers files over HTTP, payload chunking can be used to improve memory and network efficiency when transferring large amounts of data. This refers to breaking large payloads into smaller, manageable parts (chunks) for network transmission.
Many AS2 systems leverage HTTP chunked transfer encoding to implement this feature. Here’s how chunked transfer works in HTTP, which underlies AS2 communication:
AS2 Restart is a feature introduced to the AS2 protocol that allows resuming a transmission from where it left off in the event of a failure, without restarting from the beginning.
When dealing with large file transmissions, there’s always a possibility of network-level failures due to timeouts, unstable networks, or insufficient bandwidth. If the AS2 system has already transferred half of the payload, restarting from the beginning is inefficient. AS2 Restart allows the transfer to continue from the point of failure, improving efficiency. You can read more about AS2 Restart here: AS2 Restart.
AS2 systems face a unique set of challenges when dealing with very large files, but modern AS2 systems are designed to conquer them in their own ways. If you are looking for an AS2 system primarily to transfer files larger than 10 GB, it is important to ensure that the system you select is capable of handling large files and can recover quickly in case of failures. If you have more questions, please talk to one of our team members about the very large file transmission capabilities of the AS2 Gateway and MFT Gateway here.
Hirudinee has worked on a variety of products at Aayu throughout her career and has hands-on experience in planning, designing, and executing testing activities. She is the QA manager of the team with more than 7 years of experience in delivering top-notch software solutions. Outside of work, she loves traveling, trying new foods, and her dogs.
We are a US based company with Sri Lankan ingenuity majored in Cloud and Serverless technology based product engineering.
Our solutions power small businesses using our hosted SaaS solutions , to enterprises that run our software on-cloud and on-premise.
