SFTP Integration for AS2

1 Introduction

When you want to send and receive a large amount of files via the AS2 Gateway, using an integration mechanism is a must. SFTP is one such integration mechanisms currently supported by the AS2 Gateway. This section guides you on using SFTP access, for integration of AS2 communications with external systems.

1.1 What is SFTP?

SFTP stands for SSH File Transfer Protocol, or Secure File Transfer Protocol. It is a separate protocol packaged with SSH, that works similarly over a secure connection. The advantage is the ability to leverage a secure connection to transfer files and traverse the filesystem on both the local and remote system. SFTP is a good choice for integration when large numbers of files, possibly including large files, needs to be securely exchanged.

1.2 Important Information and Limits for SFTP Integration

Note: Please be aware of the following information/limitations with SFTP integration

1.2.1 Message retention in SFTP server is 48 Hours

If you enable SFTP integration, messages in your web interface will be only retained for a maximum period of 60 days. Furthermore, Attachments placed into SFTP locations are not guaranteed to be available beyond 48 hours; hence it is recommended to fetch the received files at least within 24 hours when using SFTP integration, and to delete each file from AS2 Gateway SFTP space as soon as it is retrieved. AS2 Gateway SFTP space must not be used for long-term storage of your attachments/files. See the Pricing page and the Terms page for more details.

1.2.2 Fair Usage Limitations - Concurrent connections and Throttling

To ensure fair usage, AS2 Gateway throttles its SFTP connections as follows:

• Up to 20 new connections per minute can be made from a single IP address.
• A total of up to 20 concurrent live connections can be maintained per IP address.

To save processing and bandwidth on both ends, it is recommended to:

• poll the inbox with a considerably large time period; e.g. checking for new messages once every 10 minutes.
• not maintain persistent or long-lived connections, except if/when uploading or downloading a long batch of files at a steady rate.

1.2.3 Care to be taken when Deleting files

Recursively fetching and removing all inbox content, despite being simple, can be risky - as it may:

• fetch (and then discard) files that are still being written to the SFTP filesystem (i.e. those related to messages that were received right before or during the SFTP get invocation)
• delete (lose) files for messages that were received after the recursive list/fetch was invoked (these files will not get included in the recursive fetch, but will still get deleted during the recursive removal)

Hence, when accessing SFTP programmatically, it is recommended to:

• list all files/directories in the inbox that are older than a certain time threshold (e.g. 1 minute from now)
• fetch and delete, explicitly, the content returned in the list - either as a batch (preferred), or by iterating each

1.2.4 File count and Size Limitations

You can add up to 10 files per message, adding up to a total size of 10 MB.

1.2.5 Personal Package migration to Personal with SFTP

If you are currently subscribed to the Personal package: once SFTP is activated, make sure to inform AS2 Gateway Team to get your subscription updated.

2 Enabling SFTP Integration

The AS2 Gateway runs an SFTP server and once the SFTP setup is done, users can login to the AS2 Gateway’s SFTP server to download received files and upload files to be sent.

Navigate to the ‘SFTP Feature’ page after clicking the ‘SFTP’ menu icon on the left navigation pane. If you have not already setup SFTP integration the below page will appear, allowing you to enable the integration. You can provide a username, and a pass phrase for the private key. Note that only key based SFTP access has been allowed. The username must be in latin letters (a-z, A-Z), digits (0-9), hyphen (-) and underscore (_), and cannot start with a hyphen.

Once you click on the ‘Setup SFTP’ button, what happens in the background is that a key-based authentication for SFTP will be setup for you and your private key will be made available for you to download after the setup. Note that the password which you provided during the setup will be the passphrase for your private key.

Once the SFTP setup is complete, you should be able to use any of the SFTP clients and connect to the AS2 Gateway server. After the setup completes, the SFTP view will show the instructions on connecting to the AS2 Gateway SFTP server via the command line for Linux and MacOS users. In this documentation we will cover steps on configuring connectivity using several SFTP clients including OpenSSL command line client and File Zilla and WinSCP.

A complete listing of SFTP clients can be found at https://www.sftp.net/clients

3 Connecting via SFTP

The AS2 Gateway SFTP connection information is as follows:

• Server: sftp.as2gateway.com
• Port: 9193
• Authentication: Key based

3.1 OpenSSH Command Line SFTP Client in Linux or Windows

If the sftp command is not available in your cmd, please install OpenSSH before proceeding further.

1. First download the private key file from the SFTP view.
2. Then before using the login command, we first have to set correct permissions to the private key file. Please take a look at this Super User thread of setting correct permissions to your private key file. Note: If you do not set correct permission to the private key file, you’ll get an error message.
3. Now you can run the SFTP login command as follows. Please replace with your private key file name, including the path, and replace with the SFTP username (the username which was used in the form when setting up SFTP) when executing the below command. Note that at the first time when you run the sftp login command, you will be prompted for the private key passphrase. Please provide the password which you used in the form when setting up SFTP as the passphrase.

   sftp -o Port=9193 -o IdentityFile=<private_key_file> <user_name>@sftp.as2gateway.com

3.2 WinSCP on Windows

Conversion of the Private key into Putty format

Open WinSCP, and click ‘Tools’ and then ‘Run PuTTYgen’. Then click ‘Load’ and then navigate to select the downloaded private key. It will then prompt for the passphrase. Now click ‘Save Private Key’ to save the key in Putty format.

WinSCP Site Entry

Create a new site for ‘sftp.as2gateway.com’ over port 9193, and providing your username. Click Save and create the configuration.

Now click ‘Advanced’ and specify the PPK format private key file, under SSH -> Authentication.

Now you can navigate to the /as2gateway/<station-AS2-ID>/<partner-AS2-ID> folder, to view the inbox, outbox and failed mailboxes for the Station - Partner combination.

3.3 FileZilla on Linux or Windows

FileZilla is an open source SFTP client (and Server). To configure a FileZilla client to exchange files with the AS2 Gateway, create a ‘New Site’ providing the SFTP information, and the logon type as ‘Key file’.

When you select the key file downloaded from the AS2 Gateway, it will prompt you to convert the key, and you will need to provide the passphrase for the key for this step. It will then save the PPK format file in a location selected by you.

4 SFTP Folder Structure

The SFTP integration in AS2Gateway relies on a certain folder structure to identify AS2 Stations and Partners when sending files through SFTP and when storing received file to the SFTP folder. In this section we will try to understand what this folder structure is.

Let’s start with the following sample SFTP folder structure.

As you can see in the above image, there are two directories in the top level. If you login to the SFTP server, you will see the same structure at the top level.

The .ssh folder includes configuration files related to your SFTP login, and users will not need to do any changes on that path. The folder as2gateway on the other hand is the one we should look into.

4.1 as2gateway folder

Inside the as2gateway folder, you will see zero or more folders depending on your AS2 Station configuration. If you have AS2 Stations created, you will see a folder for each AS2 station with the AS2 identifier used as the name of the folder. In the example above there are two folders for AS2 Stations with AS2 identifiers RJ_ONE and PT.

As another example, if you have stations alice, bob and partners AMZN, WLMRT, you will have the following subpaths inside as2gateway:

alice/AMZN
alice/WLMRT
bob/AMZN
bob/WLMRT

Now let’s look into the next level in the folder structure. What you see in this level are folders for each of your AS2 partners (using the partners’ AS2 identifiers as the folder names). Under each AS2 Station directory, a folder is there for each of your partners. If you look at the example structure shown above, in this case we have only one AS2 partner with the identifier DemoAS2 and you will notice that there is a folder by the name DemoAS2 under both of the Station folders PT and RJ_ONE.

Moving on to the next level, you will notice that under each AS2 partner folder, there are three folders as follows.

• failed
• inbox
• outbox

4.2 outbox folder

Out of the above mentioned folders, the most important two are the inbox folder and the outbox folder. If you want to send a file to a trading partner from a certain AS2 Station, then you should copy the file you want to be sent, to that relevant outbox folder. For an instance taking the above folder structure as a sample, say we want to send a file to DemoAS2 trading partner from the RJ_ONE AS2 station, then the path to which we should copy the file would be as2gateway/RJ_ONE/DemoAS2/outbox. So the general path to which you want to copy a file when you want to send a file via AS2 would be as follows.

as2gateway/(AS2-Station-ID)/(AS2-Partner-ID)/outbox

4.3 inbox folder

Now let’s talk about the inbox folder. The outbox folder was all about sending messages out and as the name suggests the inbox folder is all about the incoming AS2 messages. As per the same sample folder structure which we have been looking at before, if we receive a file from DemoAS2 partner to RJ_ONE station, the path which that file will be stored would be as2gateway/RJ_ONE/DemoAS2/inbox. So if we generalize that path, it would be something as follows.

as2gateway/(AS2-Station-ID)/(AS2-Partner-ID)/inbox

4.4 failed folder

failed folder will contain attachments that have been picked from outbox, but for some reason could not be enqueued for sending. If you find any files here and do not have a proper explanation regarding why the file failed, contact AS2 Gateway Team and ask for an explanation.

4.5 Inbox folder and file naming convention

You can control the path/folder structure where AS2 Gateway saves incoming files, via the Folder Structure option under your trading station:

4.5.1 Create a new timestamp-prefixed folder for each incoming message

Files for each incoming message will be saved into its own timestamp-prefixed subdirectory; i.e. under the subfolder structure:

<root dir>/
|__as2gateway/
   |__<station ID>/
      |__<partner ID>/
         |__inbox/
            |__<message processed timestamp>-<random number>/
               |__<original attachment filename>

E.g. for an invoice.edi from a message received at 2019-11-13 10:23:34.297:

as2gateway/MY_STATION/MY_PARTNER/inbox/20191113102334297-859863/invoice.edi

4.5.2 Store incoming files at inbox root, adding a timestamp-based unique prefix to its original name

In this case, there will be no per-message subfolder; instead, everything will be saved in inbox itself, with a timestamp-based unique prefix being prepended to the original attachment filename:

<root dir>/
|__as2gateway/
   |__<station ID>/
      |__<partner ID>/
         |__inbox/
            |__<message processed timestamp>-<random number>-<original attachment filename>

E.g. for an invoice.edi from a message received at 2019-11-13 10:23:34.297:

as2gateway/MY_STATION/MY_PARTNER/inbox/20191113102334297-859863-invoice.edi

4.5.3 Store incoming files at inbox root

In this case, files will be saved directly in inbox using the original filename, without any prefix or subfolder. If your partner sends multiple files with the same name, newly received files may overwrite older files; and only the latest file will remain. However, this is usually not an issue, because many partner systems use a unique prefix/suffix on each filename.

<root dir>/
|__as2gateway/
   |__<station ID>/
      |__<partner ID>/
         |__inbox/
            |__<original attachment filename>

E.g. for an invoice.edi from a message received at 2019-11-13 10:23:34.297:

as2gateway/MY_STATION/MY_PARTNER/inbox/invoice.edi

5 Test type Partners

If your trading partner is of type test (selectable at partner creation), the message folders (inbox etc.) will be nested inside another folder level test.

So if you have a test partner ALIBB, its folders under station alice would be

as2gateway/alice/ALIBB/test/inbox
as2gateway/alice/ALIBB/test/outbox
as2gateway/alice/ALIBB/test/error

If you also have defined a production partner with the same ID, ALIBB subfolder will contain the usual inbox etc. (for holding usual production data), in addition to the test subpath:

as2gateway/alice/ALIBB/inbox
as2gateway/alice/ALIBB/outbox
as2gateway/alice/ALIBB/error
as2gateway/alice/ALIBB/test/inbox
as2gateway/alice/ALIBB/test/outbox
as2gateway/alice/ALIBB/test/error

6 Sending and Receiving files via SFTP

6.1 Receiving files (AS2 message attachments) via partner inbox

When a new AS2 message is received, AS2 Gateway will persist its attachments into SFTP, under your <station-AS2-ID>/<partner-AS2-ID>/inbox folder.

The file saving path depends on the receiving trading station’s inbox folder structure config. You can fetch the file using the SFTP get command. Make sure that you also delete the fetched files, to preserve your storage quota. Deletion also eliminates the need to skip/keep track of already fetched files on your end.

cd as2gateway/<station-AS2-ID>/<partner-AS2-ID>/inbox/<timestamp>-<random number>/
get <filename> /local/file/store/path
rm <filename>
cd ..
rmdir <timestamp>-<random number>/

6.2 Sending files (messages) out via partner outbox

There are two modes of sending files out:

• To send a single file as an AS2 message, simply upload the file to the respective outbox. AS2 Gateway will pick it up and queue it for delivery, within a maximum of 30 seconds.

    put <local path>/<filename> as2gateway/<station ID>/<partner ID>/outbox/

• To send multiple files in a single AS2 message (multiple-attachment mode),

  • create a folder inside the outbox, and
  • place all files into it.

    cd as2gateway/<station ID>/<partner ID>/outbox/
    mkdir <a unique folder name; say "20200101101035">
    cd 20200101101035/
    put <file 1>
    put <file 2>
    ...
    put <file n>

AS2 Gateway will process and enqueue the folder content, two (2) minutes after the last file is uploaded. Hence, make sure that you do not leave a gap of more than 2 minutes between two consecutive file uploads.

7 Resetting SFTP

If you wish to stop using SFTP, or completely reset the SFTP space with a new keypair and clean (empty) folder structure,

• navigate to the SFTP integration page, and
• click Reset SFTP.

After reset, any previously downloaded SFTP access keys would no longer work.

If you are already subscribed and paying for the Personal package: if you do not wish to use SFTP again after the reset, make sure to inform AS2 Gateway Team to get your subscription downgraded accordingly.

If you only want to change your SFTP keypair or key passphrase, DO NOT use the reset option; instead, contact AS2 Gateway Team.