The FIX Global Technical Committee (GTC) is proud to announce the addition of all 258 Extension Packs to the FIX Latest online specification. FIX Latest is the first official release of FIX since FIX Version 5.0 Service Pack 2 (FIX 5.0 SP2) was released in 2010. FIX Latest makes available the use of all application layer messages over the FIX4 and FIXT session profiles of the FIX Session Layer standard. For the first time, the entire integrated FIX specification is available as an online format.
The transformation of the normative specification of the FIX application layer into an online format was started a few years ago, prompted by a contribution from Jim Kaye of Bank of America Merrill Lynch of an initial refactoring of the document based on the seven volume FIX specification. The FIX organization then worked to enhance the repository, which is the machine-readable version of the specification, which became FIX Orchestra. At the same time a replacement FIX website and discussion forum was developed and released that supports single sign on. The availability of FIX Orchestra and the new website infrastructure provided a platform upon which Hanno Klein, GTC Co-chair, could lead the continued refactoring and integration of the 160 Extension Packs created since the release of FIX 5.0 SP2 into an integrated online specification that is generated from a combination of markdown and FIX Orchestra, moving away from a manually edited word processing document format. This represents an incredible amount of work that has produced a considerable amount of value for the FIX user community.
Part of the restructuring included the creation of infrastructure that can automatically update the online specification, MSWord, and PDF formats of the specification with each new Extension Pack. This approach makes each enhancement to FIX Latest available to the FIX Community immediately after approval. Firms that adopt the FIX Latest branding can immediately use any of the new functionality without requiring a version change.
The first publication of the online specification (aka baseline) was made in December 2019. It was limited to the specification as defined by FIX 5.0 SP2 back in 2009, which included the first 97 Extension Packs. Whilst the content had not changed much compared to the original Volumes 3 (Pre-Trade), 4 (Trade), and 5 (Post-Trade), the generation and presentation of it had been fundamentally changed. The key difference was that a single source format (markdown) was used, from which multiple other formats (e.g. docx, html) are automatically generated with minimal manual handling effort (see below for further details). This allowed to post the specification in an interactive format to the FIX website and also provide documents for download.
The second publication in February of this year was an update to the baseline in terms of the presentation of the content, i.e. it still only provided the first 97 Extension Packs. However, the update contained component descriptions previously only available in FIXimate. It also provided images for the message structure, i.e. visualizing each message with its first level components. Navigational capabilities were enhanced to make it easier to move around across sections of the specification, both for the online format as well as for the downloadable documents.
The primary objective of this third publication has been the addition of the remaining content from the 160 Extension Packs released after FIX 5.0 SP2. This includes almost 50 new FIX messages across all areas, for example:
- Parties Reference Data
- Entitlements
- Risk Limits
- Position Transfers
- Margin Requirements
- Market Data Statistics
- Mass Orders
- Pay Management
The number of FIX fields has quadrupled from around 1500 to now almost 6000 fields. More than 3000 of these fields were added to the instrument components (includes instrument legs and underlyings) to support OTC derivative instruments, driven by the need for regulatory reporting to the CFTC in the US. The CFTC accepts FIX application messages using the FIXML encoding.
FIX Latest
FIX Latest is the most current version of FIX that supports multiple asset classes and a wide range of trading life cycle business processes. It includes all extensions and hence mitigates versioning complexities that have slowed the adoption of years of innovation and standardization of electronic trading flows.
With the introduction of FIX Latest, firms no longer need to be tied to a specific version of FIX. The FIX Community is encouraged to adopt the FIX Latest application layer messages for use over the primary FIX session layer profiles FIX4 (8=FIX.4.4) and FIXT(8=FIXT.1.1)). When an Extension Pack is approved for inclusion in the FIX standard, it can be integrated into the firm’s existing infrastructure readily, as it becomes the new FIX Latest application version. The FIX Trading Community has invested member funds into the creation of infrastructure to update and publish the online and document versions of the FIX application standard immediately following the approval of an Extension Pack. The standard is available online, as a machine-readable repository, and as a traditional downloadable document.
Behind the scenes…
The following is for those of you that are interested to know a little bit more about the approach taken by the GTC and the technologies used to automate the generation of the online spec. The approach has proven to be highly efficient and can also be applied by the FIX Trading Community to their environments whenever there are specification documents, for example rules of engagement documents for FIX interfaces. The GTC would welcome an increase in the degree of automation throughout the industry.
A fundamental design issue was to separate content from its rendering and we decided to use markdown as input format. There are many flavors of markdown and we tried to be as simple and “standard” as possible by choosing GitHub Flavored Markdown. This also has the advantage of GitHub being able to natively render repository content that is written in markdown.
This leads to the second fundamental decision, i.e. to treat the specification like source code and use a repository with all of its great features for collaboration and keeping track of who changed what, when and why. It allows to use tags to create releases of a specification, starting with a version for internal review all the way to the version for publication.
The third major decision was to use open source tools to the extent possible and to develop our own tools only where absolutely necessary. The main tool used for the transformation of the markdown content to other formats is Pandoc, a universal document converter. Pandoc was initially used to transform the existing FIX 5.0 SP2 specification from Microsoft Word (docx) to a markdown format.
Pandoc is able to use a so-called reference file (a.k.a. style file) when generating a Microsoft Word (docx) file from markdown. This reference file is a Microsoft Word (docx) file containing the styles to be used to format the markdown content. It is key to the look and feel of the resulting document and represents an important asset developed by the GTC for the FIX Trading Community. It guarantees a consistent formatting throughout the document, including the title page, table of contents and page headers and footers. The same style file is used for all application layer specification documents. The look and feel of the corresponding online versions is defined by the standard CSS (Cascading Style Sheet) of the main FIX website. It was slightly enhanced, for example to support syntax highlighting for code snippets.
Another important tool is PlantUML, an extremely powerful tool we will be using more and more for various kinds of images. It is currently being used to visualize the structure of global components in the introduction area. We have previously used it extensively for the sequence diagrams in the refactored version of the FIX Session Layer standard.
We are also using the stream editor SED, for example to replace document-internal references with URLs for the online version. We had to develop our own tool orchestra2doc to extract FIX messages, components, and fields from FIX Orchestra and generate html files for the specification appendices containing the detailed message and component layouts. Pandoc is used to transform the html files into markdown in order to use them as input for the document version of the specification, ultimately offered as a set of PDF files for download.
What’s next?
The current version of the online specification contains detailed information related to messages and components. Some of the business areas already contain more general and conceptual information specific to a business process. This will be further extended with the concepts introduced by some of the Extension Packs. There is also more refactoring work to do, for example to extract the concept of message fragmentation that is currently described more than once in the context of different messages. It is planned to introduce dedicated chapters for concepts at the beginning of each business area (pre-trade, trade, post-trade) that will only make references to messages and/or components.
The new approach for technical specifications will also be taken for the documentation of the FIX Technical Standards, i.e. to transition to markdown and to offer an online version in addition to a document for download. There is already a separate style file that will be used for the formatting and layout of all FIX Technical Standards, i.e. they will be provided in an online version later this year. The first refactoring of the FIX Technical Standards was completed for the FIX TagValue Encoding and the FIX Session Layer standard. Migration of the remaining standards is underway.
FIX Orchestra is quickly becoming the key component of any FIX infrastructure. It can be used to generate not only documentation but also encoding schemas (e.g. FIXML, SBE, JSON) or dictionaries such as QuickFIX. The GTC has created a number of public repositories in GitHub related to the development of tools for FIX Orchestra. One of them is Tablature that contains tools to convert between markdown and FIX Orchestra XML files (and vice versa). The GTC plans to use it to automatically update content stored in FIX Orchestra. We see Tablature as a key enabling approach for creating rules of engagement in a simple usable manner for the entire industry.
The tools are part of the activities of the FIX Orchestra Working Group, discussing requirements and proposals. For example, the markdown format of Tablature serves as an easy authoring format shielding from the complexities of FIX Orchestra. On the other hand, the powerful and rich feature set of the FIX Orchestra standard (scenarios, workflows, state changes etc.) is a huge benefit for the automation of software development and testing related to interfaces (not just FIX). The working group is always looking for volunteers to drive forward the development of useful tools around FIX Orchestra.
We welcome any feedback from the FIX Trading Community and have added a thread to the FIX discussion forum to engage with you on your thoughts and ideas.
Hanno Klein, GTC Co-chair EMEA
Jim Northey, GTC Co-chair Americas
Daniel Shek, GTC Co-chair Asia Pacific