HL7TransmitterService.exe is a Windows service that is used to transmit and receive HL7 messages using the Minimal Low-Level Protocol (MLLP) over TCP/IP connections. Multiple transmitters and receivers can be configured. It includes a graphical configuration program to make it easy to set up, HL7TransmitterConfig.exe.
When used in conjunction with HL7ScriptService, they can become a powerful HL7 interface engine that can accept, transform, and forward messages to and from numerous other systems.
Please note that these connections are unencrypted, plain TCP/IP. If you require secure connections (and you must, if sending or receiving PHI outside of your secure LAN), you are responsible for securing them using a VPN, SSL/TLS tunnel, or some other method. Personally, I am quite fond of stunnel.
Use the HL7TransmitterConfig.exe program to configure the service settings. There are some Global Settings that affect the entire service, like logging. Each connection is configured as either a Transmitter or a Receiver, and some settings are common to both. The configuration is stored in HL7TransmitterService.ini in the application directory, using a separate [Section] for each connection.
To add a connection, fill out the Connection Name and other properties and press Add. To edit an existing connection, select it in the list, modify the settings and press Save. To duplicate a connection, select one from the list, change the Connection Name and press Add. Use the Delete button to remove the selected connection.
The configuration program can also install and control the service. Alternately, service installation can be performed from the command line (Run as Administrator) using "HL7TransmitterService /install" (or /uninstall). The service appears in the Service Control Manager as "HL7TransmitterService".
If the settings are modified after the service has started, a service restart is required to load the new values. The config program will prompt the user if a restart is needed.
When the service is running, the "Start" button changes to "Connection Status". Pressing the button opens a dialog showing the current status, last activity time, and uptime message count for each active connection. The screen auto-refreshes to act as a live dashboard. To query the connection status info from an external process, see the Service Monitoring topic.
These settings affect the entire service.
|HL7 Transmitter Service Global Settings|
|Service ID||string||See Multiple Service Instances.|
|Log Filename||string||Log filename. May contain date substitution surrounded by %.|
|Logging Level||string||Determines how much logging is done, ranging from None to Trace.|
|Show Levels||boolean||Shows the logging level of each log entry with a single letter like [I] for Info.|
|Timestamp Format||string||Log timestamp format. Default=yyyy-mm-dd hh:nn:ss|
|Days to Keep Logs||number||Number of days to keep dated log files. (0=forever)|
If the service fails to start, check the log file. If the service is unable to use the log file specified in the ini file, it will try to write to HL7TransmitterService.log in the executable directory.
The log filename may contain a date replacement format string surrounded by percent signs (e.g. "D:\Logs\HL7Transmitter%yyyymmdd%.log"). When using a dated log file, logs older than the Days to Keep Logs setting will be cleaned up automatically at startup and each time the date changes.
The log date replacement is assumed to change no more frequently than daily, but may be longer (weekly, etc.). When using a daily log file, the suggested Timestamp Format is "hh:nn:ss" since the date is implicit for all entries in the same file.
These settings are common to both Transmitters and Receivers.
|Transmitter/Receiver Common Settings|
|Connection Name||string||The name of the connection and ini [Section]. Log entries are prefixed with this name.|
|Disabled||boolean||Disable the connection without needing to delete it. Appears as |
|Schedule||special||Determine when the transmitter or receiver will be operational.|
|Connection Log||string||Use a connection-specific log instead of the global log.|
|Connection Log Level||string||Logging level for the connection-specific log. (N=Use global level)|
|Inactivity Reset||number||Reset any TCP connections after X minutes of inactivity for "stuck" partners. (0=Off)|
|UTF-8||boolean||The sender and receiver must agree on the encoding: Default or UTF-8.|
|Port||number||The port the receiver or remote host is accepting connections on.|
A connection can be scheduled. There are three options: Always on (the default), a single start and stop time, or on a cycle. A cyclical schedule means it runs for X minutes on, then X minutes off. A cycle can be useful in situations where the host system can only accept one connection at a time, so inbound connections must take turns*. When a schedule is set, the Schedule button text will appear in bold. If a cyclical schedule is set, it will also be italicized. The button's hint will describe the current schedule. Schedule starts and stops will appear in the log at the Verbose or higher levels.
*Tip: Instead of using a schedule, just have all messages sent to HL7Transmitter, and have it collect and forward them all to the single-connection system!
Connection-specific logs can be used to put a connection's log entries into its own log instead of the global log. These can use the same date replacement syntax in the filename as the global log. All settings besides the filename and logging level come from the global log settings (timestamp format, days, etc.). A connection-specific log will not prefix each entry with the connection name since it will always be the same.
The Inactivity Reset setting should normally be left at zero (off). If the remote host exhibits behavior like getting inexplicably "stuck", unable to detect a dead connection, this can be used to completely reset any TCP connections after X minutes of inactivity (no messages sent or received) to prod the partner into re-initiating a connection.
Transmitters will connect to a remote host and periodically poll a database or directory for hl7 messages. When new messages are found, they are sent to the remote host. After positive acknowledgement, the original message is updated in the database, or the file can be deleted or moved to an archive directory for a period of time.
|HL7 Transmitter Settings|
|Remote Address||string||The IP address of the remote host.|
|Persistent Connection||boolean||When off, a connection is made only when there are messages to transmit.|
|Connect Timeout||number||Number of seconds to wait for a non-persistent connection.|
|ACK Timeout||number||Number of seconds to wait for an ACK message before giving up.|
|Polling Interval||number||Number of seconds to wait between polling for new input.|
|Transmit Delay||number||Milliseconds to pause between messages if the receiver can't handle heavy bursts.|
|Input Wildcard||string||The directory and wildcard to poll for messages to transmit.|
|Max/Interval||number||Limits the number of files processed per interval to keep the service responsive.|
|Sort By||string||Sort the file list by Time (default) or Name. Applied before Max/Interval.|
|Archive Directory||string||Path to move files to after successful processing. Blank=Delete them.|
|Archive Days||number||Number of days to keep archived files. (0=forever)|
|Fail Retries||number||Number of retries before giving up and renaming a message with the Fail Extension.|
|Fail Extension||string||Messages that get a NAK or transmit error are renamed with this extension.|
|Database Connection||string||The name of the pre-configured Database Connection.|
|Polling SQL||string||A query to poll the database for the next batch of messages that need to be sent.|
|Update SQL||string||A SQL statement to update the database with the results after a message is transmitted.|
Transmitters will maintain a persistent connection by default, keeping the TCP connection open as long as the service is running and the schedule is on. If the Persistent Connection option is turned off, a connection will be made only when there are messages to be transmitted and disconnected immediately afterwards.
Because each message must be acknowledged separately and can individually pass or fail, file-based transmitters require that each input file contains only a single message. If multi-message files need to be transmitted, use HL7ScriptService to split them into individual files for the transmitter. An example script is shown below:
<INIT> SET %OutPath = "D:\Path\Wherever\" ; The transmitter is watching this folder </INIT> HL7 COPYINPUT SET $OutFile = [UNIQUEFILENAME, %OutPath + __InName] HL7 SAVE $OutFile
If a file-based message fails to transmit and exhausts the Fail Retries limit, it will be renamed by replacing the file's original extension with the Fail Extension. If the Fail Extension contains an asterisk, it will include the file's original extension as well. For example, "BAD" would rename "foo.hl7" to "foo.BAD", and "*.BAD" would rename it to "foo.hl7.BAD".
The database Update SQL must be able to handle both failed and successful transmissions. The Database Schema and Queries section below has some suggestions on how to handle this with a single SQL statement.
The SQL editing buttons open a larger editing dialog. The dialog displays detailed help including the parameters that are available to each query. Note that line breaks in the SQL appear as \.br\ in the single-line edit and when stored in the ini. These are replaced with actual line breaks in the editor dialog and when sent to the database engine.
Receivers will listen on a port for connections and save the messages it receives to a database or a folder, one message per file. The received messages will be properly ACKnowledged to the sender.
|HL7 Receiver Settings|
|ACK MSH Template||string||Optional ACK MSH. If blank, use sender's MSH (see below).|
|ACK Time Zone (TZ)||boolean||If true, include the local time zone in the ACK's MSH.7 timestamp.|
|ACK ERR Template||string||The template for ERR segments in rejected acknowledgements (see below).|
|Connection Limit||number||Limit the number of simultaneous connections to this receiver.|
|Log Connection Info||boolean||Log the IPs and ports of the hosts that connect to the receiver.|
|Processing IDs||string||List the MSH.11 processing IDs accepted by this receiver. Blank=allow any.|
|Rules||special||Optional rules for rejecting messages with associated error codes and text.|
|Named Fields||string||A Named Fields definition to be used for validating incoming messages.|
|Validation Flags||number||What to validate: required fields, repetitions, max length, data type, and/or table.|
|File Storage Wildcard||string||The directory, filename wildcard, and extension received messages are stored under.|
|Track Dupe Control IDs||number||Track the most recent # of MSH.10 message control IDs for duplicates. (0=Off, -1=Allow blank)|
|Persist Control IDs||boolean||Persist the tracked control IDs in a file between service starts.|
|Database Connection||string||The name of the pre-configured Database Connection.|
|Duplicate Message Control ID SQL||string||SQL to check the incoming MSH.10 control ID as unique. (Blank=off, *=Allow blank).|
|Store Received Message SQL||string||A SQL statement to insert the newly received message into the database.|
A few things can cause a negative acknowledgement (NAK) to be returned on a received message, some of which are optional: the message fails to parse (bad data), the message can't be stored right now (disk full, database offline), has a missing or duplicate message control ID, has an unaccepted processing ID, fails data validation, or matches a Rule.
The first error encountered for a message determines the value returned in MSA.1. Errors in the 1xx range return "AE" (Application Error), and errors in the 2xx range return "AR" (Application Reject).
|HL7 Table 0357 - Error Codes|
|100||Segment sequence error|
|101||Required field missing|
|102||Data type error|
|103||Table value not found|
|104||Value too long|
|200||Unsupported message type|
|201||Unsupported event code|
|202||Unsupported processing id|
|203||Unsupported version id|
|204||Unknown key identifier|
|205||Duplicate key identifier|
|206||Application record locked|
|207||Application internal error|
The Processing ID option allows you to check MSH.11 to make sure you don't accept Production messages in Test and vice-versa in the event the sender misconfigures their feed. Leave it blank to accept anything. If not blank and MSH.11 appears anywhere in the value (case sensitive), the message is accepted. Typical values would be "P" for Production or "DT" for Debug/Test. If MSH.11 is blank, the message will be accepted unless the Validation Flags include Required fields.
Rules are optional HL7Script "IF" expressions that will cause a message to be rejected when evaluated as true. Each includes an error code and text that are returned to the sender in the ACK message. If the receiver has a database connection defined, it is also made available to the HL7Script used for rule processing. The Rules button will appear in bold when one or more rules have been defined.
The ACK MSH Template on receivers is used to specify an MSH segment for the ACK messages. This will be updated with the current timestamp in MSH.7 and a time-based message control id in MSH.10. The sender's trigger event is also written to MSH.9.2. If left blank (recommended), the incoming message's MSH segment will be copied for use in the ACK message, reversing the sending/receiving application/facility fields (swap MSH.3/5 and MSH.4/6), replacing the timestamp and message control ID, and changing MSH.9.1 to ACK.
The ACK ERR Template on receivers is used to specify an ERR segment layout with replaceable parameters. The ACK message will contain one such ERR segment for each error encountered in the received message. If the template contains only a single field separator (i.e. has only ERR.1), multiple errors will be sent as repetitions of ERR.1 (the old HL7 2.3 behavior). If set to blank, no ERR segments will be sent at all.
Example HL7 2.7 ERR template (default): ERR||%sid^%q^%f^%c^%s|%code|E|||||%text
Example HL7 2.3 ERR template: ERR|%sid^%q^%f^%code&%text
|ACK ERR Template Replacements|
When rejecting a message due to a Rule, the Segment ID and various indexes come from the HL7 key at the start of the rule's Error Text (if provided).
The Named Fields definition used for message validation can be read from a file or a SQL query on a database-based connection. A query must return the definition as a string in the first result field. Changes made to a Named Fields definition while the service is running won't be picked up until the service is restarted.
When storing files, the File Storage Wildcard determines the directory, any optional filename characters, asterisk, and extension. The asterisk in the wildcard is replaced with a timestamp in yymmddhhnnsszzz format and a two-digit number to guarantee a unique filename. The files are named this way to make it easy to process them in the order received. If you would like the files to be named differently, you can use HL7ScriptService to rename and/or move them based on any criteria you choose.
The SQL editing buttons open a larger editing dialog. The dialog displays detailed help including the parameters that are available to each query.
For file-based input, the Track Dupe Control IDs value indicates the number of most recently received MSH.10 message control IDs to keep in memory, with zero turning the feature off. If set to -1, blank message control IDs will be allowed (there are some senders out there that just can't follow specs). If a duplicate control ID is received, the message will be rejected. If the Persist option is checked, the IDs are saved to a file (ConnectionName_TrackDupes.txt) in the service directory at shutdown. Otherwise, the list of IDs is maintained only while the service is running.
When using database storage, the Duplicate Message Control ID SQL handles this. If the query returns zero rows, the ID is considered unique. Leave the query blank to ignore duplicates, or set it to an asterisk (*) to allow blank IDs. See Database Schema and Queries for more ideas.
Every healthcare application will have a unique way of storing HL7 messages in its database. The HL7Transmitter configuration should be flexible enough to work with just about any schema it encounters.
The HL7Tools applications do expect that each message is stored within a single field of a single record. If, for example, the application's schema were to store each segment in a different record, a view or stored procedure would need to be written to make it appear to HL7Tools as a single value.
Here, one possible message storage schema is presented along with HL7Transmitter SQL statements that work well with it. In this example, your application populates the hl7store table with the messages that are ready to be transmitted to the remote system (outbound=1/TRUE). HL7Transmitter populates the table with the messages it receives from the remote system (outbound=0/FALSE).
These examples contain syntax for both Microsoft SQL Server (MSSQL) and PostgreSQL, just two of the native drivers currently linked into HL7Transmitter. The SQL should be fairly easy to adjust to work with any other databases.
-- MSSQL CREATE TABLE hl7store ( hl7store_id integer not null identity primary key, outbound bit not null default 0, add_time datetime not null default CURRENT_TIMESTAMP, proc_time datetime, tries integer not null default 0, status nvarchar(20) not null default 'New', status_note nvarchar(max), message nvarchar(max) not null, msgid nvarchar(100), msgtime nvarchar(50), msgtype nvarchar(10), msgevent nvarchar(10), msgapp nvarchar(100), msgfac nvarchar(100), msgmrn nvarchar(100), msgacct nvarchar(100), msgbatch nvarchar(100) ) GO -- PostgreSQL CREATE TABLE hl7store ( hl7store_id serial primary key, outbound boolean not null default FALSE, add_time timestamp not null default CURRENT_TIMESTAMP, proc_time timestamp, tries integer not null default 0, status varchar(20) not null default 'New', status_note text, message text not null, msgid varchar(100), msgtime varchar(50), msgtype varchar(10), msgevent varchar(10), msgapp varchar(100), msgfac varchar(100), msgmrn varchar(100), msgacct varchar(100), msgbatch varchar(100) );
-- MSSQL SELECT TOP 250 hl7store_id, message FROM hl7store WHERE outbound = 1 AND status <> 'AA' AND status <> 'CA' AND tries < 3 ORDER BY add_time, hl7store_id -- PostgreSQL SELECT hl7store_id, message FROM hl7store WHERE outbound = TRUE AND status <> 'AA' AND status <> 'CA' AND tries < 3 ORDER BY add_time, hl7store_id LIMIT 250
This query selects the messages to transmit for the current interval. The query MUST return the message's primary key and the message itself, in that order. Any additional fields are ignored, so limit the query to the two required values if possible.
Two important considerations are the order in which messages are selected for transmission, and limiting the number of messages returned per interval to keep the service responsive. In this query, those are handled by the ORDER BY clause and the TOP/LIMIT option.
In this example, it checks for outbound messages that have a status that is neither "AA" (Application Accept) nor "CA" (Conditional Accept). Those values come from the ACK message's MSA.1 field and indicate that the message has already been successfully transmitted.
It also checks that tries is less than three, which is the database equivalent of the file-based Fail Retries/Fail Extension settings.
There are no parameters available to the Polling SQL.
-- MSSQL UPDATE hl7store SET tries = tries + 1 , proc_time = CURRENT_TIMESTAMP , status = :Status , status_note = CASE WHEN status_note IS NULL THEN '' ELSE status_note + CHAR(13) + CHAR(10) END + :MessageText WHERE hl7store_id = :PK -- PostgreSQL UPDATE hl7store SET tries = tries + 1 , proc_time = CURRENT_TIMESTAMP , status = :Status , status_note = CASE WHEN status_note IS NULL THEN '' ELSE status_note || E'\r\n' END || :MessageText WHERE hl7store_id = CAST(:PK AS integer)
The update query is responsible for updating (or possibly deleting) the message record after transmission, successful or not.
In this example, each attempt increments the tries field. If a message has been tried and failed enough times, the polling query stops picking it up.
Notice that the status_note field is appended to instead of being replaced. This keeps a history of what happened on each attempt if not successful on the first try. The status field does get replaced, always indicating the status from the most recent attempt.
This update SQL does not delete any messages from the database. It marks them as successful or failed, and at some later date a SQL job or scheduled task can remove them after they are determined to no longer be useful.
The parameters available to the update SQL are:
-- MSSQL SELECT 1 FROM hl7store WHERE outbound = 0 AND msgid = :ID -- PostgreSQL SELECT 1 FROM hl7store WHERE outbound = FALSE AND msgid = :ID
This query should look into your message store and return a simple row when the message ID has already been received. If the query returns zero rows, the ID is considered unique. If left blank, no query is run and any non-blank message ID is accepted. If set to an asterisk (*), even blank message IDs are allowed.
-- Both MSSQL and PostgreSQL INSERT INTO hl7store (message, msgid, msgtime, msgtype, msgevent, msgapp, msgfac, msgmrn, msgacct) VALUES (:MessageStr, :MSH.10, :MSH.7, :MSH.9.1, :MSH.9.2, :MSH.5, :MSH.6, :PID.3, :PID.18)
This command stores the newly received message into the database using the table's handy defaults and the following parameters:
HL7ScriptService and HL7TransmitterService can be configured to run multiple instances on a single server. A common use for this would be running both a Test and Production instance on the same server, possibly of different versions. The instructions are the same for either service.
Set up two (or more) separate application directories, each with its own copies of the executables. Run the configuration program in each directory to create the ini file. After configuring the regular settings, press the Service ID button. Enter a value that will uniquely identify the service running from each directory. Keep the value short and alphanumeric, like "Test" or "Prod".
The Service ID button will make sure your chosen ID is not already in use, and will automatically uninstall/reinstall the service as needed to change an existing ID. Any changes you may have made to the service startup type or logon account will be preserved for you. The Service ID is saved to the ini file.
When a Service ID has been assigned, the configuration program will show the ID in square brackets in the Service Control area's status text:
The service will appear in the Services management console with the Service ID value appended to the base service name. If you had set up "Test" and "Prod" instances of HL7ScriptService, you would see the entries named HL7ScriptServiceTest and HL7ScriptServiceProd.
Each service can then be configured, started, stopped, or uninstalled independently of any others.
Each instance must have unique log filenames. Attempting to share log files will cause conflicts when writing to them.