Making regulatory submissions to FDA with AS2

Making regulatory submissions to FDA (US Food and Drug Administration) includes a number of steps, and may require some custom implementations to ensure the correct structure and content of submission archive files, routing to appropriate centers, and processing and tracking of different levels/stages of acknowledgement.

Registering with FDA ESG

This is mainly the responsibility of the end user/organization, and carried out by opening a ticket in on the FDA ESG (Electronic Submission Gateway) helpdesk.

Following are the main pieces of AS2 information that FDA would request:

your AS2 identifier
your AS2, i.e. S/MIME, certificate
your AS2 receiving endpoint URL
HTTPS (TLS) certificate of above URL - or one of the chain certificates, if the leaf is short-lived

FDA usually expects your AS2 receiving endpoint to have HTTPS (TLS/SSL) security, and to be available on port 4080 of your AS2 hostname. If you do not have this, it may be advisable to set up a reverse proxy to accept HTTPS:4080 traffic and forward it to your internal AS2 system. However this is rarely required because common AS2 applications would allow you to customize the receiving port number at the application level itself.

In return, FDA will provide you with the AS2 information for their test and production environments:

AS2 identifier (usually ZZFDATST and ZZFDA)
AS2 certificates
AS2 receiving URLs
HTTPS certificate chains for above URLs

Submission archives

Archival

FDA commonly expects submissions in tar.gz (tarred and then gzipped) format.

Archives need to be well-formed, such as by having parent directory entries ahead of each file; for example, if a tarfile has a foo/bar/baz/data.xml, it should be preceded by a foo/bar/baz/ directory entry, which should in turn be preceded by a foo/bar/, and so forth. You can verify the structure using standard commands like tar -t.

// some entries...
foo/
// more entries...
foo/bar/
// more entries...
foo/bar/baz/
// even more entries...
foo/bar/baz/data.xml

Files with ordering like these, may be rejected by FDA:

// some entries...
// WRONG; missing parent foo/
foo/bar/
// more entries...
foo/bar/baz/
// even more entries...
foo/bar/baz/data.xml

// some entries...
foo/
// more entries...
// WRONG; this should have appeared after foo/bar/
foo/bar/baz/
// more entries...
foo/bar/
// even more entries...
foo/bar/baz/data.xml

// some entries...
// WRONG; missing corresponding parent directories
foo/bar/baz/data.xml

Archives should use GNU filename encoding, and STAR scheme for storing large file entries; for example, if the archive contains a file with name (path) longer than 100 characters, and something like the more common POSIX filename encoding is used, FDA will reject the file.
While this may seem strange, the actual type/format of the file (Content-Type or MIME/media-type in technical jargon) should not be included in the meta-data (MIME) headers inside the AS2 payload. Use a generic type like Content-Type: application/octet-stream - instead of a specific marker that may be normally used, such as Content-Type: application/x-gzip. (Note that the file name should still be retained properly, with the .tar.gz extension.)

In case of a rejection, you will receive a negative ACK2 which might not contain the exact cause for the rejection (other than a generic “File is corrupt: Unable to extract …” error). So it is advisable to have a standard and well-tested process to generate the archive files automatically.

File structure

Different FDA centers expect archives with different structures. Following are two common structures:

{application-ID}/{sequence-number}/{actual files}, possibly with multiple sequence-number subfolders at second level; generally used for Health Canada (HC) submissions
{application-ID}/{sequence-number}/{actual files}, but with only a single sequence-number subfolder; generally used for non-HC eCTD submissions

Transmission and routing

FDA relies on HTTP headers sent on the incoming AS2 request, to route your submissions to the appropriate center. FDA has documented these routing headers in detail, and the primary headers are as follows:

X-Cyclone-Metadata-FdaSubmissionType: type of the submission
X-Cyclone-Metadata-FdaCenter: intended recipient, i.e. FDA center

To improve efficiency, uniformity and reduce chance of errors, it is recommended to create a set of submission profiles for each FDA center-submission combination, so that you can quickly submit an archive to the appropriate center without entering/tuning the headers each time.

FDA systems are known to have problems with special characters in HTTP headers, so it is generally recommended to stick to the basic ASCII character set (alphanumerics and a limited set of symbols) in otherwise flexible headers such as the message Subject. Submissions with malformed headers may receive a transmit error (HTTP error response code) from FDA.

Handling acknowledgements (ACKs)

FDA returns at least 2 ACKs for each submission, depending on the receiving center and document type. Each contains increasingly detailed feedback regarding the submission.

Most commonly, there are three ACKs:

ACK1 (MDN)

This is the basic AS2-level receipt or confirmation, indicating whether the original submission file reached FDA’s AS2 system without issues. It covers the main security aspects of the AS2 protocol: confidentiality, integrity and non-repudiation.

ACK2

This indicates whether the submission was routed to the appropriate center and decoded/extracted successfully, within FDA infrastructure. It also contains the correlation ID (commonly known as CoreID) that FDA has assigned for the submission, for use in processing any further ACKs (as described in the next section).

The file usually takes plain-text format (.txt) or HTML (.html), and contains the CoreID in its name as well.

ACK3

This contains details of the final processing result, and any acceptance/rejection details of the overall submission.

It is named using the CoreID (plus the file-type extension), and may come in different formats including XML, HTML and even PDF - depending on the recipient center.

You may receive ACK1 either as a new AS2 transaction (if the original FDA-bound transmission is tuned to use asynchronous MDNs - which is usually the case), or as the response of the original transmission itself (as a synchronous MDN). ACK2,ACK3 and so forth, are always new AS2 transactions/transmissions initiated from FDA side.

Correlating ACKs with the original submission

All ACKs beyond ACK2, only contain the FDA-assigned CoreID as the sole tracking mechanism for the overall submission. Without knowing the CoreID, the sender has no way to determine which returned ACK file corresponds to which original submission/file.

The ACK2 file, usually contains 2 lines similar to the following: the original AS2 identifier (Message-ID) of the submission message you sent, and the CoreID that FDA assigned for it. This is the key mapping that drives the tracking process.

MessageId:<original-as2-id@acme.com>

CoreId: ci00timestampms0.12345@fdslv67890_te1 

(Note that if ACK2 is in HTML format, these lines may contain extraneous characters; however the HTML-parsed (“readable”) content would still be the same.)

Now, any further ACKs (which only contain the CoreID, within their filename), can be traced back to the original submission as:

filename → CoreID → original Message-ID from ACK2 → original submission

It is recommended to maintain an easily accessible trail, linking the submission and all received ACKs, so that transactions can be reviewed, filtered, searched and easily analyzed for auditing, tracking and troubleshooting purposes.

Putting it all together: The Regulatory Submissions Platform

We offer AS2 Gateway for Regulatory Submissions, a ready-made dedicated product that offers all of the above features - in addition to all facilities of the AS2 Gateway core product, and several other value additions:

a dedicated submissions dashboard where you can track all your submissions, easily and conveniently - without the complexities of underlying AS2 protocols or messaging internals
maintain separate submission profiles for different FDA centers, combining the routing headers, source/target AS2 identities, and submission archive generation formats
directly mount a data volume to pick submissions from a predefined folder structure, instead of having to manually create or upload archives from the web interface
get notified automatically about submission status and progress (receipt of ACKs, which may in some cases take several hours)
optional, direct access to underlying AS2 transactions/transmissions and errors, whenever required, for fine-grained troubleshooting

In addition to FDA, the Regulatory Submissions solution also supports submissions to other regulatory entities like EMA (European Medicines Agency), with its own set of archival rules and specialized ACK processing support. In addition, it can also be used to submit regular AS2 messages (or pre-composed reguatory submission files) to any standard trading entity/partner over AS2, similar to the SaaS platform or the On-premise product.

The solution is capable of performing VLF (very large file) submissions up to several gigabytes in size, efficiently and transparently, using AS2 restart capabilities of the core AS2 engine.

The Regulatory Submissions solution is built on top of AS2 Gateway technology, with a strong track record of 10+ years of leading the AS2 B2B communications space. All hosting options available for the On-premise product - containerized/orchestrated, virtual machines, bare-metal, etc. - are available for the Regulatory Submissions product as well.

Contact the AS2 Gateway Team for more details and a demo/PoC installation.