HL7TransmitterService.exe is a Windows service that is used to transmit and receive HL7 messages using the Minimal Low-Level Protocol (MLLP) over TCP/IP connections. Multiple transmitters and receivers can be configured. It includes a graphical configuration program to make it easy to set up, HL7TransmitterConfig.exe.

When used in conjunction with HL7ScriptService, together they become a powerful HL7 interface engine that can acccept, transform, and forward messages to and from numerous other systems.

Please note that these connections are unencrypted, plain TCP/IP. If you require secure connections (and you must, if sending or receiving PHI outside of your secure LAN), you are responsible for securing them using a VPN, SSL/TLS tunnel, or some other method. Personally, I am quite fond of stunnel.

Table of Contents

Setup and Configuration

Download HL7Tools here

Use the HL7TransmitterConfig.exe program to configure the service settings. The settings are stored in HL7TransmitterService.ini in the application directory. Each connection that you configure is either a Transmitter or a Receiver, and is stored in a separate [Section] in the ini file.

The configuration program can also install and control the service when Run as Administrator (which it does by default). Alternately, you can use "HL7TransmitterService /install" (or /uninstall) in a command prompt that was started using "Run as administrator". The service appears in the Service Control Manager as "HL7TransmitterService".

If you modify the settings after the service has started, you must restart the service to load the new values. The config program will prompt you to do this.

When the service is running, the "Start" button changes to "Connection Status". Press the button to open a dialog showing the current status, last activity time, and uptime message count for each active connection.

HL7TransmitterConfig

Return to Top

Global Settings

HL7 Transmitter Service Global Settings
Setting NameTypeNotes
Service ID string See Multiple Service Instances.
Log Filename string Log filename. May contain date substitution surrounded by %.
Logging Level string Determines how much logging is done, ranging from None to Debug.
Show Levels booleanShows the logging level of each entry into the log with a single letter like [I] for Info.
Timestamp Format string Log timestamp format. Default=yyyy-mm-dd hh:nn:ss
Days to Keep Logsnumber Number of days to keep dated log files. (0=forever)

If the service fails to start, check the log file. If the service is unable to use the log file specified in the ini file, it will try to write to HL7TransmitterService.log in the executable directory.

The log filename can contain a date replacement format string surrounded by percent signs (e.g. "D:\Logs\HL7Transmitter%yyyymmdd%.log"). When using a dated log file, logs older than the Days to Keep Logs setting will be cleaned up automatically at startup and each time the date changes.

The log date replacement is assumed to change no more frequenly than daily, but may be longer (weekly, etc.). When using a daily log file, the suggested Timestamp Format is "hh:nn:ss" since the date is implicit for all entries in the same file.

Connection-specific logs can be used to put a connection's log entries into its own log instead of the global log. These can use the same date replacement syntax in the filename as the global log. All settings besides the filename and logging level come from the global log settings (timestamp format, days, etc.). A connection-specific log will not prefix each entry with the connection name since it will always be the same.

A connection can be scheduled. There are three options: Always on (the default), a single start and stop time, or on a cycle. A cyclical schedule means it runs for X minutes on, then X minutes off. A cycle can be useful in situations where the host system can only accept one connection at a time, so you have to take turns with another transmitter. When a schedule is set, the Schedule button text will appear bolded. If a cyclical schedule is set, it will also be italicized. The button's hint will describe the current schedule. Schedule starts and stops will appear in the log at the Verbose or Debug levels.

Return to Top

Transmitters

Transmitters will connect to a remote host and periodically poll a database or directory for hl7 messages. When new messages are found, they are sent to the remote host. After positive acknowledgement, the original message is updated in the database, or the file can be deleted or moved to an archive directory for a period of time.

HL7 Transmitter Settings
Setting NameTypeNotes
Connection Name string The name of the connection and ini [Section]. Log entries are prefixed with this name.
Disabled booleanDisable the connection without needing to delete it. Appears as strikethrough.
Schedule specialDetermine when the transmitter will connect to the remote host. See above.
Connection Log string Use a connection-specific log instead of the global log.
Connection Log Level string Logging level for the connection-specific log. (N=Use global level)
UTF-8 booleanThe sender and receiver must agree on the encoding: Default (single-byte) or UTF-8.
Port number The port the remote host is accepting connections on.
Remote Address string The IP address of the remote host.
Persistent ConnectionbooleanWhen off, a connection is made only when there are messages to transmit.
Connect Timeout number Number of seconds to wait for a non-persistent connection.
ACK Timeout number Number of seconds to wait for an ACK message before giving up.
Polling Interval number Number of seconds to wait between polling for new input.
Transmit Delay number Milliseconds to pause between messages if your receiver can't handle heavy bursts.
File-based
Input Wildcard string The directory and wildcard to poll for messages to transmit.
Archive Directory string Path to move input files to after successful processing. Blank=Delete them.
Archive Days number Number of days to keep archived files. (0=forever)
Fail Retries number Number of retries before giving up and renaming a message with the Fail Extension.
Fail Extension string Messages that get a NAK or transmit error are renamed with this extension.
Max/Interval number Limits the number of files processed per interval to keep the service responsive.
Database
Database Connectionstring The name of the pre-configured Database Connection.
Polling SQL string A query to poll the database for the next batch of messages that need to be sent.
Update SQL string A SQL statement to update the database with the results after a message is transmitted.

Transmitters will maintain a persistent connection by default, keeping the TCP connection open as long as the service is running and the schedule is on. If the Persistent Connection option is turned off, a connection will be made only when there are messages to be transmitted and disconnected immediately afterwards.

If a file-based message fails to transmit and exhausts the Fail Retries limit, it will be renamed by replacing the file's original extension with the Fail Extension. If the Fail Extension contains an asterisk, it will include the file's original extension as well. For example, "BAD" would rename "foo.hl7" to "foo.BAD", and "*.BAD" would rename it to "foo.hl7.BAD".

The database Update SQL must be able to handle both failed and successful transmissions. The Database Schema and Queries section below can give you an idea how to handle this with a single SQL statement.

The SQL editing buttons open a larger editing dialog that includes detailed help including the parameters that are available to each query. Note that line breaks in your SQL appear as \.br\ in the single-line edit and when stored in the ini. These are replaced with actual CRLF line breaks in the editor dialog and when sent to the database engine.

Return to Top

Receivers

Receivers will listen on a port for connections and save the messages it receives to a database or a folder, one message per file. The received messages will be properly ACKnowledged to the sender.

HL7 Receiver Settings
Setting NameTypeNotes
Connection Name string The name of the connection and ini [Section]. Log entries are prefixed with this name.
Disabled booleanDisable the connection without needing to delete it. Appears as strikethrough.
Schedule specialDetermine when the receiver will accept connections. See above.
Connection Log string Use a connection-specific log instead of the global log.
Connection Log Level string Logging level for the connection-specific log. (N=Use global level)
UTF-8 booleanThe sender and receiver must agree on the encoding: Default (single-byte) or UTF-8.
Port number The port to listen for connections on.
ACK MSH Template string Optional ACK MSH. If blank, use sender's MSH (see below).
ACK ERR Template string The template for ERR segments in rejected acknowledgements (see below).
Connection Limit number Limit the number of simultaneous connections to this receiver.
Log Connection Info booleanLog the IPs and ports of the hosts that connect to the receiver.
Processing IDs string List the MSH.11 processing IDs accepted by this receiver. Blank=allow any.
Named Fields File string A Named Fields file to be used for validating incoming messages.
Validation Flags number What to validate: required fields, repetitions, max length, data type, and/or table.
File-based
File Storage Wildcard string The directory, optional prefix, and extension received messages are stored under.
Track Dupe Control IDsnumber Track the most recent # of MSH.10 message control IDs for duplicates. (0=Off, -1=Allow blank)
Persist Control IDs booleanPersist the tracked control IDs in a file between service starts.
Database
Database Connection string The name of the pre-configured Database Connection.
Duplicate Message Control ID SQLstring SQL to check the incoming MSH.10 control ID as unique. (Blank=off, *=Allow blank).
Store Received Message SQL string A SQL statement to insert the newly received message into the database.

A few things can cause a negative acknowledgement (NAK) to be returned on a received message, some of which are optional: the message fails to parse (bad data), the message can't be stored right now (disk full, database offline), has a missing or duplicate message control ID, has an unaccepted processing ID, or fails data validation.

The Processing ID option allows you to check MSH.11 to make sure you don't accept Production messages in Test and vice-versa in the event the sender misconfigures their feed. Leave it blank to accept anything. If not blank and MSH.11 appears anywhere in the value (case sensitive), the message is accepted. Typical values would be "P" for Production or "DT" for Debug/Test. If MSH.11 is blank, the message will be accepted unless the Validation Flags include Required fields.

The ACK MSH Template on receivers is used to specify an MSH segment for the ACK messages. This will be updated with the current timestamp in MSH.7 and a time-based message control id in MSH.10. The sender's trigger event is also written to MSH.9.2. If left blank (recommended), the incoming message's MSH segment will be copied for use in the ACK message, reversing the sending/receiving application/facility fields (swap MSH.3/5 and MSH.4/6), replacing the timestamp and message control ID, and changing MSH.9.1 to ACK.

The ACK ERR Template on receivers is used to specify an ERR segment layout with replaceable parameters. The ACK message will contain one such ERR segment for each error encountered in the received message. If the template contains only a single field separator (i.e. has only ERR.1), multiple errors will be sent as repetitions of ERR.1 (the old HL7 2.3 behavior). If set to blank, no ERR segments will be sent at all.

Example HL7 2.7 ERR template (default): ERR||%sid^%q^%f^%c^%s|%code|E|||||%text

Example HL7 2.3 ERR template: ERR|%sid^%q^%f^%code&%text

ACK ERR Template Replacements
ValueReplaced With
%codeError code
%textError text
%sidSegment ID
%qSegment sequence
%fField index
%cComponent index
%sSubcomponent index
%rRepetition index

Changes made to a Named Fields file while the service is running won't be picked up until the service is restarted.

When storing files, the File Storage Wildcard determines the directory, optional prefix, and extension. The asterisk in the wildcard is replaced with a timestamp in yymmddhhnnsszzz format and a two-digit number to guarantee a unique filename. The files are named this way to make it easy to process them in the order received. If you would like the files to be named differently, you can use HL7ScriptService to rename and/or move them based on any criteria you choose.

The SQL editing buttons open a larger editing dialog that includes detailed help including the parameters that are available to each query.

For file-based input, the Track Dupe Control IDs value indicates the number of most recently received MSH.10 message control IDs to keep in memory, with zero turning the feature off. If set to -1, blank message control IDs will be allowed (there are some senders out there that just can't follow specs). If a duplicate control ID is received, the message will be rejected. If the Persist option is checked, the IDs are saved to a file (ConnectionName_TrackDupes.txt) in the service directory at shutdown. Otherwise, the list of IDs is maintained only while the service is running.

When using database storage, the Duplicate Message Control ID SQL handles this. If the query returns zero rows, the ID is considered unique. Leave the query blank to ignore duplicates, or set it to an asterisk (*) to allow blank IDs. See Database Schema and Queries for more ideas.

Return to Top

Database Connections

The HL7Tools applications are built with Delphi, which comes with the fantastic FireDAC components to allow for connectivity to numerous database systems. FireDAC provides the FDAdministrator.exe application for configuring these connections, which is distributed with HL7Tools.

The HL7Tools applications present a drop-down list of previously configured connection definitions. Pressing the "..." button next to a Database Connection entry will launch the FDAdministrator application so you can create new connections or edit existing ones.

By default, FireDAC first looks in the application directory for the FDConnectionDefs.ini and FDDrivers.ini files. The config programs will create empty versions of those files for you if the ini files and FireDAC registry keys for the workstation default location are not already present.

FDAdministrator

IMPORTANT: In order to save a connection definition, you MUST click on the [+] in the Connection Definitions list to verify/test the connection. An unsuccessful connection will not be saved.

Additional help for FDAdministrator can be found here: http://docwiki.embarcadero.com/RADStudio/Tokyo/en/FDExplorer

HL7Tools includes the following drivers. If your database of choice is not currently supported, please provide feedback.

My apologies: The FDAdministrator application doesn't quite live up to my usual "no surprises" user experience standards, but I don't think writing a replacement would be the best use of my time. It works well enough, and is a set-it-and-forget-it sort of tool.

Return to Top

Database Schema and Queries

Every healthcare application will have a unique way of storing HL7 messages in its database. The HL7Transmitter configuration should be flexible enough to work with just about any schema it encounters.

Here, one possible message storage schema is presented along with HL7Transmitter SQL statements that work well with it. In this example, your application populates the hl7store table with the messages that are ready to be transmitted to the remote system (outbound=1/TRUE). HL7Transmitter populates the table with the messages it receives from the remote system (outbound=0/FALSE).

These examples contain syntax for both Microsoft SQL Server (MSSQL) and PostgreSQL, just two of the native drivers currently linked into HL7Transmitter. The SQL should be fairly easy to adjust to work with any other databases.

-- MSSQL
CREATE TABLE hl7store (
  hl7store_id integer not null identity primary key,
  outbound bit not null default 0,
  add_time datetime not null default CURRENT_TIMESTAMP,
  proc_time datetime,
  tries integer not null default 0,
  status nvarchar(20) not null default 'New',
  status_note nvarchar(max),
  message nvarchar(max) not null,
  msgid nvarchar(100),
  msgtime nvarchar(50),
  msgtype nvarchar(10),
  msgevent nvarchar(10),
  msgapp nvarchar(100),
  msgfac nvarchar(100),
  msgmrn nvarchar(100),
  msgacct nvarchar(100),
  msgbatch nvarchar(100)
)
GO

-- PostgreSQL
CREATE TABLE hl7store (
  hl7store_id serial primary key,
  outbound boolean not null default FALSE,
  add_time timestamp not null default CURRENT_TIMESTAMP,
  proc_time timestamp,
  tries integer not null default 0,
  status varchar(20) not null default 'New',
  status_note text,
  message text not null,
  msgid varchar(100),
  msgtime varchar(50),
  msgtype varchar(10),
  msgevent varchar(10),
  msgapp varchar(100),
  msgfac varchar(100),
  msgmrn varchar(100),
  msgacct varchar(100),
  msgbatch varchar(100)
);

Transmitter Polling SQL

-- MSSQL
SELECT TOP 250 hl7store_id, message
FROM hl7store
WHERE outbound = 1 AND status <> 'AA' AND status <> 'CA' AND tries < 3
ORDER BY add_time, hl7store_id

-- PostgreSQL
SELECT hl7store_id, message 
FROM hl7store
WHERE outbound = TRUE AND status <> 'AA' AND status <> 'CA' AND tries < 3
ORDER BY add_time, hl7store_id
LIMIT 250

This query selects the messages to transmit for the current interval. The query MUST return the message's primary key and the message itself, in that order. Any additional fields are ignored, so limit the query to the two required values if possible.

Two important considerations are the order in which messages are selected for transmission, and limiting the number of messages returned per interval to keep the service responsive.

In this example, it checks for outbound messages that have a status that is neither "AA" (Application Accept) nor "CA" (Conditional Accept). Those values come from the ACK message's MSA.1 field and indicate that the message has already been successfully transmitted.

It also checks that tries is less than three, which is the database equivalent of the file-based Fail Retries/Fail Extension settings.

There are no parameters available to the Polling SQL.

Transmitter Update SQL

-- MSSQL
UPDATE hl7store SET tries = tries + 1
, proc_time = CURRENT_TIMESTAMP
, status = :Status
, status_note = CASE WHEN status_note IS NULL THEN '' ELSE status_note + CHAR(13) + CHAR(10) END + :MessageText
WHERE hl7store_id = :PK

-- PostgreSQL
UPDATE hl7store SET tries = tries + 1
, proc_time = CURRENT_TIMESTAMP
, status = :Status
, status_note = CASE WHEN status_note IS NULL THEN '' ELSE status_note || E'\r\n' END || :MessageText
WHERE hl7store_id = CAST(:PK AS integer)

The update query is responsible for updating (or possibly deleting) the message record after transmission, successful or not.

In this example, each attempt increments the tries field. If a message has been tried and failed enough times, the polling query stops picking it up.

Notice that the status_note field is appended to instead of being replaced. This keeps a history of what happened on each attempt if not successful on the first try. The status field does get replaced, always indicating the status from the most recent attempt.

The parameters available to the update SQL are:

:PK
The message's primary key from the Polling SQL as a string. If the primary key isn't actually a string, typecast the value if your database requires it.
:ACK
A boolean/bit indicating whether an ACK was received or not.
:Status
The value of MSA.1 when :ACK is true, otherwise a word indicating what happened, like "Timeout" or "Error".
:MessageStr
When :ACK is true, the entire ACK message. Otherwise, a possibly longer description of what went wrong, like "ACK timeout" or "HL7 Error".
:MessageText
The same as :MessageStr, but when it's the ACK message the CR segment terminators are replaced with CRLF.
HL7 Keys
When :ACK is true, you can use any valid HL7 key as a parameter to retrieve string values from the ACK message, such as :MSA.1 or :ERR.3. Keys not present in the message simply return an empty string ('').

Receiver Duplicate Message Control ID SQL

-- MSSQL
SELECT 1 FROM hl7store WHERE outbound = 0 AND msgid = :ID

-- PostgreSQL
SELECT 1 FROM hl7store WHERE outbound = FALSE AND msgid = :ID

This query should look into your message store and return a simple row when the message ID has already been received. If the query returns zero rows, the ID is considered unique. If left blank, no query is run and any non-blank message ID is accepted. If set to an asterisk (*), even blank message IDs are allowed.

Available parameters:

:ID
The MSH.10 message control ID as a string.

Receiver Store Message SQL

-- Both MSSQL and PostgreSQL
INSERT INTO hl7store (message, msgid, msgtime, msgtype, msgevent, msgapp, msgfac, msgmrn, msgacct)
VALUES (:MessageStr, :MSH.10, :MSH.7, :MSH.9.1, :MSH.9.2, :MSH.5, :MSH.6, :PID.3, :PID.18)

This command stores the newly received message into the database using the table's handy defaults and the following parameters:

:MessageStr
The full message with standard CR segment terminators.
:MessageText
The full message with CRLF segment terminators.
HL7 Keys
Any valid HL7 key may be used as a parameter to retrieve string values from the received message, such as :MSH.10 or :PID.3. Keys not present in the message simply return an empty string ('').

Return to Top

Multiple Service Instances

HL7ScriptService and HL7TransmitterService can be configured to run multiple instances on a single server. A common use for this would be running both a Test and Production instance on the same server, possibly of different versions. The instructions are the same for either service.

Set up two (or more) separate application directories, each with its own copies of the executables. Run the configuration program in each directory to create the ini file. After configuring the regular settings, press the Service ID button. Enter a value that will uniquely identify the service running from each directory. Keep the value short and alphanumeric, like "Test" or "Prod".

The Service ID button will make sure your chosen ID is not already in use, and will automatically uninstall/reinstall the service as needed to change an existing ID. Any changes you may have made to the service startup type or logon account will be preserved for you. The Service ID is saved to the ini file.

When a ServiceID has been assigned, the configuration program will show the ID in square brackets in the Service Control area's status text:

HL7 Service ID

The service will appear in the Services management console with the ServiceID value appended to the base service name. If you had set up "Test" and "Prod" instances of HL7ScriptService, you would see the entries named HL7ScriptServiceTest and HL7ScriptServiceProd.

Each service can then be configured, started, stopped, or uninstalled independently of any others.

Each instance must have its own unique log filename(s); they must not try to share a common log file or they will fight over who gets to write to it.

Return to Top