SFTP Integration for AS2 & SFTP
1 Introduction
The MFTGateway allows SFTP based integration with your existing systems, in a very similar manner to the native AWS S3 integration. The location paths are the same as for S3, since the MFT Gateway actually utilizes Amazon Transfer service over the same AWS S3 bucket created for each account, to expose the SFTP connectivity.
This section guides you on using SFTP access, for integration of AS2 & SFTP communications with external systems.
1.1 What is SFTP?
SFTP stands for SSH File Transfer Protocol, or Secure File Transfer Protocol. It is a separate protocol packaged with SSH, that works similarly over a secure connection. The advantage is the ability to leverage a secure connection to transfer files and traverse the filesystem on both the local and remote system. SFTP is a good choice for integration when large numbers of files, possibly including large files, needs to be securely exchanged.
1.2 Enabling SFTP Integration
Navigate to ‘SFTP’ after clicking the ‘Integrations’ menu icon on the left navigation pane. If you have not already setup SFTP integration the above page will appear, allowing you to enable the integration. You can provide a username, and choose to either generate a new key, or opt-to login using an existing SSH key that you already have. Only key based SFTP access has been allowed. Note that the username must be in latin letters (a-z, A-Z), digits (0-9), hyphen (-) and underscore (_), and cannot start with a hyphen.
To generate a new key, choose the Private key type desired, based on how you plan to access the SFTP server, and your client operating system. You can optionally add a password to protect the generated private key. This will be your passphrase to open the private key file for later use.
Windows based systems
The Putty format keys (PPK) works best for Windows systems. When using WinSCP as an example, download the
👉 Read this article to learn how to log into the MFT Gateway SFTP server from FileZilla SFTP Client
Linux based systems
If you created a new key, download it to the client system. First you need to change file permissions to only allow read access to the user. Then you can connect by providing the username, and the key file.
sudo chmod 400 <private-key>
sftp -i <private-key> <username>@sftp.mftgateway.com
If you choose to ‘enable with existing key pair’, you will need to copy the public key of your existing keypair. Usually this will be located as .ssh/id_rsa.pub file, on a Linux based operating system.
$ cat ~/.ssh/id_rsa.pub
ssh-rsa AAA...QLWY5Ow== user@host
1.3 SFTP Folder Structure in AS2 & SFTP Directories
1.3.1 SFTP Folder Structure in AS2 Directory
Once you login via SFTP, you will be placed into your home directory, which is mftg-<your-mftgateway-account-id>. The following hierarchy exists for each account. <Station-AS2-ID> refers to the AS2 ID of the local Station defined in the MFT Gateway, while <Partner-AS2-ID> refers to the AS2 ID of the remote partner. The AS2/raw-message contains the encrypted AS2 messages as-sent, or as-received, while the AS2/files contains the actual payload files without, or after decryption. It should be noted that AS2/keystore and AS2/tmp are special locations and end-users should not access them, and are listed here only for completeness.
AS2/files[Raw files of messages]<Station-AS2-ID>/<Partner-AS2-ID>/inbox<Station-AS2-ID>/<Partner-AS2-ID>/outbox
AS2/headers[Raw HTTP headers of messages]<Station-AS2-ID>/<Partner-AS2-ID>/inbox<Station-AS2-ID>/<Partner-AS2-ID>/outbox
AS2/keystore[Key stores]https.jks[SSL/TLS certificates and keypairs]identity.jks[AS2 Station encryption/signature certificates]trust.jks[AS2 Partner certificates]
AS2/raw-mdn[Raw MDNs]incomingoutgoing
AS2/raw-message[Raw AS2 messages]<Station-AS2-ID>/<Partner-AS2-ID>/inbox<Station-AS2-ID>/<Partner-AS2-ID>/outbox
AS2/send[Send location]AS2/tmp[Temporary file location]
1.3.2 SFTP Folder Structure in SFTP Directory
Similar to the AS2 directory found in the home directory, mftg-<your-mftgateway-account-id>, we can find the SFTP directory for SFTP communications. The hierarchy in the SFTP directory is as follows. SFTP/external is the external directory that is accessible by your SFTP partner when you partner connects to the SFTP server. The directory SFTP/partners contains the messages that have been sent and received by each SFTP partner. Note that <Partner-SFTP-ID> refers to the SFTP ID of the remote partner. And the SFTP/send directory is used to send the actual SFTP messages to the relevant partner.
SFTP/external[Location accessible by SFTP partners]<Partner-SFTP-ID>/inbox<Partner-SFTP-ID>/outbox<Partner-SFTP-ID>/send
SFTP/partners[Access messages sent to & received by partners]<Partner-SFTP-ID>/inbox<Partner-SFTP-ID>/outbox
SFTP/send[Send location]<Partner-SFTP-ID>/
1.4 Disabling SFTP
You may disable SFTP access, create a new keypair, or provide a new key, by visiting the SFTP integration page and disabling the current SFTP integration.
2.2 Default Lifecycle rules
MFT Gateway utilizes Amazon S3 lifecycle rules to manage transactional objects stored. Objects stored under the following directories will move to non-current status after 35 days, and will be permanently deleted after another 35 days. i.e. Objects will be deleted 70 days since creation.
| AS2 | SFTP |
|---|---|
| AS2/files | SFTP/external |
| AS2/raw-message | SFTP/partners |
| AS2/raw-mdn | |
| AS2/headers |
Objects stored in AS2/tmp directory will be moved into non-current status after 1 day, and will be permanently deleted after 2 days from the creation.
3 Automation via SFTP over AS2 Protocol
3.1 Sending Files
To submit a file to be sent over AS2, copy (i.e. SFTP put) it to the location AS2/send/<station-AS2-id>/<partner-AS2-id>/. If you want to send a single file, you can copy it directly to this location. If you have more than one file to be submitted as a single AS2 message, zip the files into a single archive, and then upload the archive.
e.g
put <local-file> AS2/send/<station-AS2-id>/<partner-AS2-id>/<attachment-name>
After a file has been submitted for send, the system will be automatically triggered, and will perform the AS2 communications process to send the file to the designated partner. The processed file will be placed into the AS2/files/<Station-AS2-ID>/<Partner-AS2-ID>/outbox location
Files uploaded over AS2 with the same name within a 6 hour window will be considered as duplicates and will not be processed. If you need to submit files with the same name, add a postfix to the filename, such as an extension (Duplicated files will be moved to
AS2/duplicate/<Station-AS2-ID>/<Partner-AS2-ID>/)
SFTP triggers only apply for the content directly uploaded to the above mentioned directory. Any content copied or moved within the SFTP folder structure itself, will be ignored
3.2 Receiving Files
Received files will be persisted to the AS2/files/<station-AS2-id>/<partner-AS2-id>/inbox. You can fetch/download these payload files through SFTP get.
e.g
ls AS2/files/<station-AS2-id>/<partner-AS2-id>/inbox/
get AS2/files/<station-AS2-id>/<partner-AS2-id>/inbox/<timestamp-random-number>/<attachment-name> <local-path>
Note that, by default, each message will have a unique path, with the timestamp followed by a random number, within which the actual attachments will be found.
You can customize the file structure by updating the AS2 Partner’s File Structure settings.
3.3 Accessing MDNs, sent files, HTTP headers and raw messages
If required, the MDN’s sent and received, files successfully sent, raw (encrypted) messages or message headers can be downloaded from the SFTP locations as listed in the folder structure layout.
4 Automation via SFTP over SFTP Protocol
4.1 Sending Files
To submit a file to be sent over SFTP, copy (i.e. SFTP put) it to the location SFTP/send/<Partner-SFTP-ID>/. If you want to send a single file, you can copy it directly to this location. If you have more than one file to be submitted as a single SFTP message, zip the files into a single archive, and then upload the archive.
e.g
put <local-file> SFTP/send/<Partner-SFTP-ID>/<attachment-name>
After a file has been submitted for send, the system will be automatically triggered, and will perform the SFTP communications process to send the file to the designated partner. The processed file will be placed in the SFTP/partners/<Partner-SFTP-ID>/outbox location
Files uploaded over SFTP with the same name within a 6 hour window will be considered as duplicates and will not be processed. If you need to submit files with the same name, add a postfix to the filename, such as an extension (Duplicated files will be moved to
AS2/duplicate/<Station-AS2-ID>/<Partner-AS2-ID>/)
SFTP triggers only apply for the content directly uploaded to the above mentioned directory. Any content copied or moved within the SFTP folder structure itself, will be ignored
4.2 Receiving Files
Received files will be persisted to the location SFTP/partners/<Partner-SFTP-ID>/inbox. You can fetch/download these payload files through SFTP get.
e.g
ls SFTP/partners/<Partner-SFTP-ID>/inbox/<random-message-id>/
get SFTP/partners/<Partner-SFTP-ID>/inbox/<random-message-id>/<attachment-name> <local-path>
Note that, by default, each message will have a unique path, with the timestamp followed by a random number, within which the actual attachments will be found.
You can customize the file structure by updating the SFTP Partner’s File Structure settings.