MIME, S/MIME and AS2
This section is relevant to those who are interested to learn and educate themselves about the internal operation of the AS2 protocol. It is not applicable for end users, or those still new to AS2.
1 Introduction to MIME
Multipurpose Internet Mail Extensions (MIME) defined by RFC2045, 2046 and 2047 is a standard allowing different types of content, to be properly encoded and represented. Email uses MIME extensively, and any message written, usually includes a text only version, as well as an HTML version of the content, as shown in the example below.
1.1 MIME Parts
The different content pieces in MIME are called ‘parts’ or ‘MIME parts’. Each MIME part has a ‘Content-Type’ which defines the content it holds, such as ‘text/plain’ or ‘text/html’. Each part has a unique ‘boundary’ to indicate the start and end of it, when multiple parts exist together. The ‘Content-Type’ of such a multiple part content is ‘multipart/
To: someone@somewhere.com
From: another@somewhereelse.com
Subject: Welcome to MIME
Message-ID: <7649a04d-1466-3493-e8a7-7c5084d12285@adroitlogic.com>
Date: Sun, 15 Aug 2021 23:00:30 +0530
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="------------F601116DC026CBBB6BCDDB62"
--------------F601116DC026CBBB6BCDDB62
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Hello everyone, this is a MIME message
--------------F601116DC026CBBB6BCDDB62
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
<html>
<body>
<p>Hello everyone, this is a <b>MIME</b> message</p>
</body>
</html>
--------------F601116DC026CBBB6BCDDB62
1.2 Binary parts and encoding
Similarly, binary data such as an image can be represented as follows, with the actual content being encoded as binary, 8Bit, or Base64 (shown below). A Content-Disposition header can optionally specify a file name for the content. This allows the content to be written to a file with the intended / original name after decoding.
Content-Type: image/jpeg
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=flowers.jpg
iQCVAwUBMJrRF2N9oWBghPDJAQE9UQQAtl7...HOxEa44b+EI=
1.3 Content Transfer Encoding
The ‘Content-Transfer-Encoding’ specifies the encoding of the actual content, which could be one of several values. These are defined in Section 5 of the RFC 1341 and can be summarized as follows:
7 bit
The data consists only of the US-ASCII characters, which uses the lower 7 bits for each character. With each line being 1,000 characters or fewer. Contents are easily readable by a human.
This is a 7 bit encoded text with only US-ASCII characters. Line length will be less than 1,000 characters.
8 bit
The data can contain extended ASCII characters; using the 8th bit to indicate special characters. Each line is 1,000 characters or fewer.
Hi Tom<C3><A1><C5><A1>
The above encoded 8 bit binary content will correctly render as follows.
Hi Tomáš
Binary
This is same as the 8 bit, except that there’s no line length restriction. See example above for 8 bit.
Quoted-printable
Allow lines greater than 1,000 characters by encoding line breaks and creating a result with a line length not exceeding 76 characters. Also allow non US-ASCII characters to be used by escaping them. Any 8-bit value may be encoded with three characters: an ‘=’ followed by two hexadecimal digits representing the byte’s numeric value. While ‘=0A’ is a meaningful line break, ‘=SPACE’ is a soft line break, and ‘=0A0D’ a hard line break.
e.g. Note the encoding of ‘=’ as ‘=3D’, the line breaks at 76 characters and the encoding of the words ‘Antoine de Saint-Exupéry’
... <table cellspacing=3D"0" cellpadding=3D"0" style=3D"width:100%;border-botto= m:1px><tr><td>Antoine de Saint-Exup=C3=A9ry</td</tr> ...
Base64
This allows binary content to be encoded into US-ASCII characters, with line lengths not exceeding 76 characters. In essence, 3 raw bytes encode as 4 US-ASCII characters. The following example shows the text ‘Hello MIME’ when encoded as base64.
SGVsbG8gTUlNRQo=
2 Secure/Multipurpose Internet Mail Extensions (S/MIME)
S/MIME defined by RFC8551 provides capabilities to digitally sign and encrypt contents, in addition to compression. While digital signatures provide authentication, message integrity, and non-repudiation with proof of origin, Encryption provides data confidentiality.
2.1 How does an S/MIME signed message look?
The following example shows how a signed EDI 997 functional acknowledgement will look when digitally signed. Note that it is possible to view the original content being signed as a MIME part, with the signature appearing as another MIME part. In the following example, the signature includes an SHA-256 digest. The validation of a digital signature assures the receiver that the signed MIME part has not been tampered with during transit. It also provides a guarantee that the content originated from the signing party.
Content-Type: multipart/signed; micalg=sha-256;
boundary="--------------F601116DC026CBBB6BCDDB62--";
protocol="application/pkcs7-signature"
--------------F601116DC026CBBB6BCDDB62--
Content-Type: application/edi-x12
Content-Disposition: Attachment; filename=fa997.txt
ISA*00* *00* *01*007912345 *ZZ*ACPISA
*210721*1038*;*00501*110000098*0*P*>~
GS*FA*007912345*ACPGS*20210721*1038*220000098*X*005010~
ST*997*330000098~AK1*SH*200200013~AK2*856*300300013~AK5*A~
AK9*A*1*1*1~SE*6*330000098~GE*1*220000098~IEA*1*110000098~
--------------F601116DC026CBBB6BCDDB62--
Content-Type: application/pkcs7-signature; name=smime.p7s
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=smime.p7s
MIIBJgYJKoZIhvcNAQcCoIIBFzCCARMCAQExADALBgkqhkiG9w0BBwExgf4w
... truncated ... Mv5jQEDiwVX5DRasxu6Ld8zv9u5/TsdBNiufGw==
--------------F601116DC026CBBB6BCDDB62--
2.2 How does an Encrypted S/MIME message look?
Encrypted content is held in an S/MIME envelope as follows, and contains the binary or base64 version of the payload after encryption. Thus, the original content is not visible to any eavesdropper, and cannot be tampered either. A signed message will look similar the following once its encrypted. After decryption, it will contain the original S/MIME form of a signed content as shown in the earlier example.
Content-Type: application/pkcs7-mime; name=smime.p7m; smime-type=enveloped-data
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=smime.p7m
MIIBHgYJKoZIhvcNAQcDoIIBDzCCAQsCAQAxgcAw ... truncated ... XeCSFOSOpOJ7FuVyU=
2.3 Compression of MIME parts
Compression maybe used over the source content, before security (i.e. signing and encryption), or after security (i.e. signing and encryption). The ‘Content-Type’ of a compressed mime part will indicate ‘smime-type=compressed-data’. RFC 5402 ‘Compressed Data in an Internet EDI Message’ defines compression. The following example shows a compressed signature part.
Content-Type: application/pkcs7-mime; smime-type=compressed-data;
name=smime.p7z
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=smime.p7z
MIAGCyqGSIb3DQEJEAEJoIAwgAIBADANBgsqhkiG9w0BCRADCDCABgkqhkiG9w0BBwGg
... truncated ...fU5ivWnasml72XFb3gb5druui7ytN803M570nii7C5r8tfwR281
3 Dissection of an AS2 transmission
Considering S/MIME above and HTTP/S as the transport mechanism, we can now understand how an AS2 message will appear over the network. AS2 uses HTTP POST method and several HTTP headers called AS2 headers such as ‘AS2-From’ and ‘AS2-To’, indicating the AS2 identifiers of the sender and receiver. A unique ‘Message-Id’ is assigned to each message, and a set of appropriate S/MIME headers have been used to describe the content.
The following sample shows an encrypted, signed payload, which is the most commonly used configuration. It is possible for the payload to be unencrypted but signed, or even unencrypted and unsigned, although those options are not common.
When creating an AS2 message to be sent, the sender computes a hash of the payload, called the Message Integrity Check (MIC). The sender saves the computed MIC, and the Message-ID assigned to the outgoing message. The sender then signs the message using its private key, and then encrypts it using the receiver’s public key / certificate. The AS2 message is sent using an HTTP POST request, after adding AS2-To, AS2-From and other headers.
The receiver then decrypts the received message, after validating the AS2 headers. It will then validate the signature placed by the sender, using the public key/certificate of the sender. This verifies the authenticity and origination of the message, and guarantees that it has not been tampered. The sender, after processing the message, creates an MDN response. This is sent as the HTTP response when a synchronous MDN is requested, or as a new HTTP request, if an asynchronous MDN was requested. The MDN is composed by including the Original-Message-ID and computed MIC, in addition to other information, and includes human readable and machine readable components.
HTTP POST: http://service.mftgateway.com/receiver
AS2-From: SENDER_AS2_ID
AS2-To: RECEIVER_AS2_ID
AS2-Version: 1.1
Content-Transfer-Encoding: binary
Content-Type: application/pkcs7-mime; name="smime.p7m"; smime-type=enveloped-data
Disposition-Notification-Options: signed-receipt-protocol=required,
pkcs7-signature; signed-receipt-micalg=required,sha1
Disposition-Notification-To: asankha@gmail.com
Message-Id: <520887901042033@mftgateway.com>
MIME-Version: 1.0
Subject: AS2 message from MFT Gateway
[omitted binary encrypted data]
3.1 Processing of the received message
Receiver uses its private key to decrypt the received message. This reveals the signed message contents as shown below. The receiver then verifies the signature, using the senders public certificate which has been shared with the receiver, previously by the sender. This ensures that the receiver has received the message intact from the now authenticated sender.
Content-Type: multipart/signed; micalg=sha-256;
boundary="--------------F601116DC026CBBB6BCDDB62--";
protocol="application/pkcs7-signature"
--------------F601116DC026CBBB6BCDDB62--
Content-Type: application/edi-x12
Content-Disposition: Attachment; filename=fa997.txt
ISA*00* *00* *01*007912345 *ZZ*ACPISA
*210721*1038*;*00501*110000098*0*P*>~
GS*FA*007912345*ACPGS*20210721*1038*220000098*X*005010~
ST*997*330000098~AK1*SH*200200013~AK2*856*300300013~AK5*A~
AK9*A*1*1*1~SE*6*330000098~GE*1*220000098~IEA*1*110000098~
--------------F601116DC026CBBB6BCDDB62--
Content-Type: application/pkcs7-signature; name=smime.p7s
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=smime.p7s
MIIBJgYJKoZIhvcNAQcCoIIBFzCCARMCAQExADALBgkqhkiG9w0BBwExgf4w
....................Mv5jQEDiwVX5DRasxu6Ld8zv9u5/TsdBNiufGw==
--------------F601116DC026CBBB6BCDDB62--
3.2 Generation of the MDN
The sender specifies if the submission requires a receipt, through the ‘Disposition-Notification-Options’ header. If ‘signed-receipt-protocol’ value is ‘required’ then a receipt is mandatory, while a value of ‘optional’ leaves the decision to the receiver to decide. The ‘signed-receipt-micalg’ additionally dictates the use of the expected hash algorithm such as SHA1 or SHA-256, and if using the specified algorithm is mandatory. If the ‘Disposition-Notification-To’ specifies an HTTP/S URL, then it implies that the receipt must be sent as a new HTTP/S message. Otherwise the Message Disposition Notification (MDN) is sent as the HTTP/S response for the submission. The former is requesting an ‘asynchronous MDN’, while the latter is requesting a ‘synchronous MDN’.
Listing below depicts a sample MDN being sent, which is signed by the receiver, using the SHA256 hash algorithm. The receiver has signed the MDN, and this outer payload is of the content type ‘multipart/signed’. Within this the ‘multipart/report’ content is the MDN. The MDN contains a ‘human readable’ part as plain text, which states a summary of the MDN, along with a ‘message/disposition-notification’ part, which is the ‘machine readable’ MDN. This contains the received Message Integrity Check (MIC) computed by the receiver, along with the hash algorithm used.
Message-Id: <520888026290198@mftgateway.com>
MIME-Version: 1.0
Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha256;
boundary="----=_Part_1_1564170442.1629925602750"
------=_Part_1_1564170442.1629925602750
Content-Type: multipart/report; report-type=disposition-notification;
boundary="----=_Part_0_871065432.1629925602628"
------=_Part_0_871065432.1629925602628
Content-Type: text/plain
Content-Transfer-Encoding: 7bit
MDN for Message-ID: <520887901042033@mftgateway.com>
From: RECEIVER_AS2_ID
To: SENDER_AS2_ID
Received on: Wed Aug 25 21:06:42 UTC 2021
Status: processed
Comment: This is not a guarantee that the message has been completely processed
or understood by the receiving translator. Powered by MFT Gateway
------=_Part_0_871065432.1629925602628
Content-Type: message/disposition-notification
Content-Transfer-Encoding: 7bit
Reporting-UA: MFT Gateway
Original-Recipient: rfc822; RECEIVER_AS2_ID
Final-Recipient: rfc822; RECEIVER_AS2_ID
Original-Message-ID: <520887901042033@mftgateway.com>
Received-Content-MIC: snqVfv/aolEVibuExwgerihU93M=, sha1
Disposition: automatic-action/MDN-sent-automatically; processed
------=_Part_0_871065432.1629925602628--
------=_Part_1_1564170442.1629925602750
Content-Type: application/pkcs7-signature; name=smime.p7s; smime-type=signed-data
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature
[omitted binary pkcs7 signature data]
------=_Part_1_1564170442.1629925602750--
3.3 HTTP Headers used in AS2 messages
The table below describes the HTTP headers used in AS2 communications, such as AS2 messages, MDNs and HTTP responses.
Header Name | AS2 Message | MDN | Use / Sample values |
---|---|---|---|
AS2-Version | Mandatory | Mandatory | 1.0 / 1.1 or 1.2 |
AS2-From | Mandatory | Mandatory | AS2 identifier of the sender |
AS-To | Mandatory | Mandatory | AS2 identifier of the receiver |
Message-Id | Mandatory | Mandatory | Unique message ID |
Disposition-Notification-To | Mandatory | N/A | MDN sync/async ? e.g. email@org.com for sync or URL such as http://org.com/async_mdn for async |
Disposition-Notification-Options | Mandatory | N/A | MDN options e.g. signed-receipt-protocol=optional, pkcs7-signature; signed-receipt-micalg=optional,sha1 |
Receipt-Delivery-Option | Optional | N/A | Async MDN URL such as http://org.com/async_mdn |
Content-Type | Mandatory | Mandatory | Depends on message ‘application/pkcs7-mime’ for encrypted, ‘multipart/signed’ for signed AS2 or MDN |
Content-Transfer-Encoding | Optional | N/A | usually present with encrypted AS2 messages |
Content-Disposition | Optional | N/A | usually present with encrypted AS2 messages |
MIME-Version | Optional | Optional | usually value 1.0 |
Subject | Optional | Optional | e.g. AS2 Message from MFT Gateway (as used in email) |
From | Optional | Optional | e.g. email of sender (as used in email) |
Date | Optional | Optional | e.g. Thu, 19 Dec 2002 15:03:38 GMT (as used in email) |
EDIINT-Features | Optional | N/A | e.g. multiple-attachments or CEM |
3.4 MIME Content-Type’s and uses in AS2
multipart/signed
A digitally signed MIME part, specifying the MIC algorithm, protocol and boundary for the content being signed e.g.
multipart/signed; micalg=sha1; protocol="application/pkcs7-signature"; boundary="------=_Part_111--"
The ‘boundary’ separates the source content being signed, and the actual signature computed over it
Content-Type: multipart/signed; boundary="------=_Part_111--"; protocol="application/pkcs7-signature"; micalg=sha1 ------=_Part_111-- Content-Type: text/plain Content-Transfer-Encoding: 7bit ` This is the source content being signed ------=_Part_111-- Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=smime.p7s ` MIIBJg.... << This is the signature >> ......XMDJL4A5h8== ------=_Part_111--
application/pkcs7-signature
A digital signature part in binary or base64 etc, as shown in the above example
multipart/report
An MDN response part, containing a human understandable ‘text/plain’ part, and a machine readable ‘message/disposition-notification’ part.
Content-Type: multipart/report; report-type=disposition-notification; boundary="----=_Part_0_682182987.1629953546489" ` ------=_Part_0_682182987.1629953546489 Content-Type: text/plain Content-Transfer-Encoding: 7bit ` MDN for Message-ID: <521167388627070@mftgateway.com> From: ACP_PROD To: RECEIVER_AS2_ID Received on: Thu Aug 26 04:52:26 UTC 2021 Status: processed Comment: This is not a guarantee that the message has been completely processed or understood by the receiving translator. Powered by MFT Gateway ` ------=_Part_0_682182987.1629953546489 Content-Type: message/disposition-notification Content-Transfer-Encoding: 7bit ` Reporting-UA: MFT Gateway Original-Recipient: rfc822; RECEIVER_AS2_ID Final-Recipient: rfc822; RECEIVER_AS2_ID Original-Message-ID: <521167388627070@mftgateway.com> Received-Content-MIC: snqVfv/aolEVibuExwgerihU93M=, sha1 Disposition: automatic-action/MDN-sent-automatically; processed ------=_Part_0_682182987.1629953546489--
message/disposition-notification
The machine readable part of an MDN, including the original message ID, the computed content MIC. See example above.
application/pkcs7-mime
An encrypted MIME part, usually in binary or base64. e.g. application/pkcs7-mime; smime-type=enveloped-data; name=smime.p7m
text/plain
A plain text content part. e.g. the human understandable part of an MDN
application/edi-x12
An American Standards Committee X12 EDI message
3.5 Multiple attachments in AS2
The IETF RFC 6362 defines ‘Multiple Attachments for Electronic Data Interchange - Internet Integration (EDIINT)’. The essence of this RFC is that existing MIME technology can be used to send a set of files within one AS2 message. When more than one file needs sending, each is converted into a MIME part, and then accumulated under a single ‘multipart/related’ MIME part. This part can be processed through compression, signatures and encryption as usual. The following is an example of a multipart/related MIME payload containing an XML document, and a PDF file. The RFC states that compatible AS2 software implementing support for multiple attachments must include an HTTP header ‘EDIINT-Features: multiple-attachments’ into the AS2 request.
Content-type: multipart/related; boundary="------=_Part_0_682182987.1629953546489--";
start="<root.attachment>"; type="application/xml"
------=_Part_0_682182987.1629953546489--
Content-type: application/xml
Content-ID: <root.attachment>
<employee id="12345"><name>Asankha Perera</name><department>IT</department></employee>
------=_Part_0_682182987.1629953546489--
Content-type: application/pdf
Content-ID: <2nd.attachment>
[PDF DOCUMENT]
------=_Part_0_682182987.1629953546489--
3.6 Filename preservation in AS2
An IETF draft defines ‘Filename Preservation for EDIINT’. Simply put, this specification states that the source filename should be maintained when transmitted through AS2. Filenames, allowed lengths, and allowed characters of a filename, are usually system dependent. However, the source filename can be easily transmitted within the same MIME part carrying the contents of the file, by simply specifying a ‘Content-Disposition’ header with the filename. The receiving party can then use this filename, when the payload is persisted back into a file.
Content-Type: application/edi-x12
Content-Disposition: Attachment; filename=fa997.txt
ISA*00* *00* *01*007912345 *ZZ*ACPISA
*210721*1038*;*00501*110000098*0*P*>~
GS*FA*007912345*ACPGS*20210721*1038*220000098*X*005010~
ST*997*330000098~AK1*SH*200200013~AK2*856*300300013~AK5*A~
AK9*A*1*1*1~SE*6*330000098~GE*1*220000098~IEA*1*110000098~
3.7 Certificate Exchange or CEM in AS2
An IETF draft defines ‘Certificate Exchange Messaging for EDIINT’, to allow parties to exchange certificate information using an XML message schema. It is required for AS2 software supporting this specification to use the HTTP header ‘EDIINT-Features: CEM’. The objective of this specification was the automatic rotation of expiring certificates. Although the XML schema allowed detailed information for exchange, in practice organizations would almost always notify other partners of rotated certificates, well before expiry.