Orchestra Tutorials – FIX Trading Community

Extension Packs now with Orchestra XML File

Hanno Klein — Tue, 07 Apr 2020 16:26:34 +0000

The FIX Global Technical Committee (GTC) is proud to announce the availability of a FIX Orchestra XML file with every Extension Pack (EP) starting with EP255. The file follows Version 1.0 of the new FIX Technical Standard FIX Orchestra published in February 2020 (https://www.fixtrading.org/standards/fix-orchestra/). It is a metadata standard that defines all elements of a messaging application as well as the syntax for the rules that govern these elements. The term FIX Orchestra file stands for the physical representation of an electronic interface that follows the FIX Orchestra standard and uses the XML format.

Objective

The FIX Orchestra file published with every EP contains all FIX messages, components, repeating groups, fields and values (aka FIX Latest). It does not use any of the extended features of FIX Orchestra to define subsets, scenarios, workflows, etc. as it simply represents the complete repository.

The FIX Trading Community can use it as a central source and Golden Record of information for any of their FIX interfaces. It is intended for development and testing environments and to be consumed by applications generating artifacts such as specifications, source code and test cases.

Extension Pack Content

Every EP (https://www.fixtrading.org/extension-packs/) contains a number of individual files as follows:

ASBUILT: PDF document with the Gap Analysis showing only the incremental extensions to FIX together with the newly assigned tag numbers and values
NEW Orchestra: ZIP file with the representation of FIX Latest (complete application layer) using the new Technical Standard FIX Orchestra Version 1.0
Repository: ZIP file with the two legacy representations of FIX Latest (complete application layer), i.e. basic repository and unified repository (aka FIX Repository 2010 Edition)
FIXML Schema: ZIP file with a set of XML files representing the FIXML encoding of FIX Latest (complete application layer)

FAQ

Question	Answer
What is the difference between the FIX Orchestra file published by FIX and FIX Orchestra files I exchange with counterparties to define our Rules of Engagement?	There is no difference in structure, i.e. both are based on the Technical Standard FIX Orchestra Version 1.0. The main differences are in the scope of the content and the number of Orchestra features used. The FIX Orchestra file published by FIX as part of an EP does not use advanced features such as scenarios or workflows. These may be relevant when publishing an Orchestra file together with recommended practices having a narrow business focus. A FIX Orchestra file representing Rules of Engagement between two counterparties is only a small subset of the FIX Orchestra file published by FIX. However, it will contain additional elements such as user defined fields or values and may use advanced features to describe the desired behavior.
What is the difference between the FIX Orchestra file and the FIX Repository file published by FIX?	The content is identical in terms of the content, i.e. covering the complete application layer (FIX Latest). However, the structure of the files is quite different. The FIX GTC has developed a tool to migrate an existing unified repository to a FIX Orchestra file. Please contact the FPL Program Office if you are looking into the transition to FIX Orchestra.
How long will the legacy representations of FIX Latest (basic and unified repository) be published alongside the new representation with FIX Orchestra?	The legacy representations of FIX Latest (basic and unified repository) will continue to be part of an EP as long as it is required to support existing applications in the community. We encourage the community to get involved with FIX Orchestra to discover the benefits of a transition. The FIX GTC is also working on a number of tools making it easier to get started with FIX Orchestra, also if a FIX repository had not been used before.
Where can I learn how to use FIX Orchestra?	The documents describing the Technical Standard are available here. If you are a FIX member, you can join the FIX Orchestra Working Group to find out about available tools to get you started. The FIX Trading Community also has a number of public projects on GitHub. This also includes Orchestra tutorials.

Concepts-Part5-Service-Offerings-And-Session-Configurations

FIXTrading Community — Wed, 11 Sep 2019 17:57:28 +0000

Orchestra Concepts Part 5:

Service Offerings and Session Configurations

The Orchestra Interfaces schema provides a standardized way to specify service offerings and configure sessions for connectivity.

Overview

Every system that allows customers or partners to connect uses some stack of communications protocotols. Those protocols must be specified to the peers in order to establish connectivity. We call an application-layer protocol a service offering. A firm may have several service offerings for different business phases, such as order routing, market data, reference data, and post-trade clearing and settlement. The Orchestra Interfaces schema provides a standardized way to specify those APIs.

Commonly, order entry customers are directed to a dedicated gateway by a listening address and port, and sessions are assigned specific identifiers. In FIX, sessions are traditionally identified by a combination of SenderCompID and TargetCompID, but other protocols have different identifiers. It is now common to use public/private key infrastructure to securely identify parties. The Orchestra Interfaces schema provides features for transport connectivity and user identification, either by declared IDs or with security keys.

Service Offerings

Each service offering in an Orchestra Interfaces file is specified by an element under the root . The application layer protocol for a service offering is named in a element. A service is configured with an orchestration, another Orchestra file in the repository schema. It contains the message structures and workflow associated with that service in formats described by parts 1 and 2 of this tutorial.

An orchestration file may be distributed to peers in a bundle with an interfaces file or may be accessible through a web inteface.

Protocol Stack

Below application layer, a service runs over a protocol stack that includes a session protocol, message encoding, transport, and possibly other protocols. At each layer, a protocol version and configuration may be given.

Here’s a sample interface definition with two orchestrations. It all works over TagValue encoding, FIXT session layer, and TCP transport.

Session Configuration

For each service offering, a party may be assigned one or more sessions. A session configuration includes session identifiers, transport settings, and possibly security keys. Each / element is contained by an that describes its service offering.

Here’s and example of a session configuration:


    
    
    
    
    
    
        XYZ
    
    
        ABC
    
    
    2019-08-07T09:30:00Z

A element can contain configuration for any protocols in the stack that is specific to the session. In this case, the transport is always TCP for the service offering, but each session contains transport settings that are unique to that session. In this example, a primary and secondary port are offered for fail-over.

Optionally, security keys may be provided for a session in a element. It must follow the textual encoding format described in IETF RFC 7468. Optional parts include certificates and private keys.

Concepts-Part5-Service-Offerings-And-Session-Configurations

FIXTrading Community — Thu, 29 Aug 2019 19:52:18 +0000

Orchestra Concepts Part 5:

Service Offerings and Session Configurations

The Orchestra Interfaces schema provides a standardized way to specify service offerings and configure sessions for connectivity.

Overview

Service Offerings

An orchestration file may be distributed to peers in a bundle with an interfaces file or may be accessible through a web inteface.

Protocol Stack

Here’s a sample interface definition with two orchestrations. It all works over TagValue encoding, FIXT session layer, and TCP transport.

Session Configuration

Here’s and example of a session configuration:


    
    
    
    
    
    
        XYZ
    
    
        ABC
    
    
    2019-08-07T09:30:00Z

Optionally, security keys may be provided for a session in a element. It must follow the textual encoding format described in IETF RFC 7468. Optional parts include certificates and private keys.

Concepts-Part4-Actors-And-External-States

FIXTrading Community — Thu, 29 Aug 2019 19:50:29 +0000

Orchestra Concepts Part 4:

Actors and External States

Messages convey information from one participant to another. Their actions and responses also depend upon persistent state information that can be defined in Orchestra.

Overview

The participants in a service play different roles. In FIX, the terms Party and PartyRole are very common. With respect to a trade, a party may play the role of executing firm, clearing firm, client, contra firm, and so forth.

In Orchestra, we abstract the role of a party that sends or recieves messages as an actor. As explained in Part 2: Workflow and Scenarios, actors send and receive messages on flows and the exchange of messages can be defined in Orchestra using workflow features. Specifically, a message may trigger one of several possible responses, and each response can have a conditional expression that tells when that response is fired.

Responses do not depend solely the contents of messages. The response to an order also depends whether the market is open, on other orders resting in a book, and possibly whether a limit price exceeds some daily bracket. The TradeDate field in an ExecutionReport also depends on state information of the market (not always the same as calendar date). Orchestra supports external states in two ways: state variables of actors and state machines.

State Variables

State variables are persistent data items associated with an actor that are external to messsages. To be clear, like message structures, Orchestra defines the format of the data, not actual instances of data. It is up to a system implementation to store data in memory or in a persistent database, etc. Orchestra defines interfaces and rules of behavior, not implementation details.

To keep it simple, Orchestra uses the same syntax to define state variables as it does to define the elements of messages. A stand-alone data item is a field, but components and repeating groups are also supported.


    
    
        
            Repeating group of security status by SecurityID

In this example, “Market” is the name of an actor that receives orders and sends executions and other messages. Its first state variable is defined as . Without showing its full definition, understand that each instance of the group tells the current trading status of an instrument. In a live system,this would be held in some kind of database (possibly all cached in memory), but Orchestra provides an abstraction for the data structure with standardized semantics.

So far, the state variables reference an existing FIX field and repeating group. It is also possible to define state elements that don’t correspond to existing message elements. To do that, define a directly under the using a user-defined tag.

State variables become useful when a conditional expression depends on their values. Not shown here, but a message response force an update of an actor’s state variable. Perhaps an order updates the current best bid or offer, for example.

State Machines

Many interactions in financial systems are described as transitions between discreet states. A formal model of states and their transitions is called a state machine. A great example of a state machine in FIX is OrdStatus. That field has code set that corresponds to the possible states of an order. Some transitions are valid, such as New (initial state) to Partially Filled, but others are disallowed, such as Canceled to Filled. Canceled is called a terminal state; once an order is canceled, it remains dead forever.

Here’s an example of a state machine for market phases:

In this example, there is an initial state called Closed. The other states are Halted, Open, Preopen, and Preclose. From Closed, the only valid transition is Reopening, which results in the Preopen state, and so forth. Notice that Open state has two possible transitions; one goes to Halted and the other goes to Preclose.

Here’s what the state machine looks like in standard UML format:

Other features can be added to a state machine that are not illustrated here for simplicity.

A transition can trigger an action such as sending a message. For example, when the market is opened, a session status message can be sent from one actor to another. Conversely, a message response can invoke a state machine transition such as an order state change.
A conditional expression can be used to guard a transition so it is only allowed under certain circumstances.

to come

Back

Orchestra Concepts Part 3: Conditional Expressions

Concepts-Part2-Workflow-and-Scenarios

FIXTrading Community — Thu, 29 Aug 2019 19:43:17 +0000

Orchestra Concepts Part 2:

Workflow and Scenarios

In addition to message structures, an Orchestra file can also convey the workflow, or expected message exchange behaviors of a service offering. Scenarios are the key to Orchestra’s power.

Overview

A session in FIX or other financial industry protocol involves an exchange of application messages of various types (aside from the messages of the underlying session and transport protocols). The expected message exchange behaviors are important part of the rules of engagement between counterparties. To use a service offering, a counterparty must send the right kind of message for the circumstances.

Some messages are part of a synchronous request/response pattern while others are asynchronous or unsolicited. Overall, we call these application behavioral patterns workflow. The Orchestra XML schema provides the means to express what kinds of responses are possible to a message, and when each possibility applies.

In FIX, message types are overloaded for meaning. An ExecutionReport is sent when when an order is booked, when it executes immediately— either partially or fully, when it is rejected, and so forth. The message layouts of ExecutionReport events are slightly different for each use case. For instance, an execution conveys a trade quantity and price, while a rejection conveys a reject reason instead. Each use case of a message type is known in Orchestra as a scenario. Unlike legacy data dictionaries, Orchestra provides a way to specify each scenario individually rather than as a conglomeration of all its use cases.

What’s it good for?

Receiving machine-readable definitions of message exchange behaviors can be used for …

Generating stubs of application code
Writing messaging test cases and validations
Generating documentation for humans

Actors and Flows

An actor plays a role with regard to a system described by an Orchestra file. For example, in an order routing system, one actor would be “trader” and another would be “market”. An element is defined for each role in an Orchestra file.

A element represents a stream of messages from one actor (source) to another (destination). Often, a pair of flows constitute a conversation between actors. For example, orders flow from trader to market while executions flow back from market to trader.

Here’s an example with three flows between actors. As you can see, flows can have various attributes, but we won’t dive into those details right now since we want to concentrate on message exchange. (You can refer to the Orchestra specification or XML schema for details.) We’ll also show in a later tutorial how actors can have state information that influence messaging behaviors.

Message Workflows

Each message defined in an Orchestra file belongs to a flow. From its flow attribute, you can infer the source and destination of the message from the object of the same name.

Here’s an example of an order message on an “OrderEntry” flow, which is sent from BuySide to Market.

In the previous tutorial, we explored message structures. The NewOrderSingle message definition would contain its members within a element as we showed before, but we’ll skip that for the moment because now we want to explore behaviors.

A sibling to the element is the element where workflow is defined. The parent element may contain any number of elements.

For now, we’ll concentrate only on sending message responses, but be aware that Orchestra also can specify triggering of other kinds of events as well, including timers and actor state changes. (See the XML schema for all forms.) Each response that results in sending a message contains a element.

Here’s a simplified view of the possible message responses to a NewOrderSingle message:


    
        
        
        $Market.SecMassStatGrp[SecurityID==in.SecurityID].SecurityTradingStatus != ^TradingHalt and $Market.Phase == "Open"
    
    
        
        $Market.Phase == "Closed"
    
    
        
        
        !exists $Market.SecMassStatGrp[SecurityID==in.SecurityID]"
    
    
        
        
        $Market.SecMassStatGrp[SecurityID==in.SecurityID].SecurityTradingStatus != ^TradingHalt and $Market.Phase == "Open"
        
            One or more fills may occur; the first one may be synchronous.

Here are some important elements and attributes to notice:

Each of the elements contains the identifiers of a message that must be defined elsewhere in the Orchestra file. That is, the id of the messageRef must match the id of a corresponding element. That’s where you would find the of the response message.
Every message and messageRef has a scenario attribute. If not provided, its value is “base” by default. To restate the point above more precisely, a messageRef matches a message definition by combination of its id and scenario attributes. That combination defines a specific use case of a message type. The structure of ExecutionReport message of scenario rejected is different than scenario traded, and so forth. Not all the responses need be the same message type, as shown by the response of BusinessMessageReject.
Each of the responses shown here also contains a element. That element contains a Score DSL expression that must evaluate to true or false (called a predicate in formal logic). If it evaluates to true, then its response is triggered. If there is no element in a response, then it is triggered unconditionally.
Responses can have other attributes such as a minimum or maximum number of times it can occur. For example, implMinOccurs="0" implMaxOccurs="1" on the acknowledgement response says that it can occur at most once for a given order but may never occur. By default, there is no limit to the number of times a response can trigger.
Also, responses can be specified as synchronous or asynchronous. The fill response is specified as sync="asynchronous" since the timing of a trade is not determined by the time that the order was entered into the market. It could occur seconds, minutes or hours later (if ever).

Here’s how the workflow looks as a UML sequence diagram:

Not shown in the example, a response can also tell how the key identifiers of a response message relate to the original message. For example, an ExecutionReport generally carries the ClOrdId of the original order message.

Message Scenarios

Let’s compare to of the responses in the example above but with some details stripped out.

The two responses both refer to a response message and both of them are ExecutionReport messages. The events are not the same though, and although both responses have msgType="8", the messasge layouts are not the same. Both ExecutionReport messages have the usual identifiers, only a trade produces a price and quantity while an order acknowledgement does not. Likewise, and ExecutionReport for order expiration or rejection would contain different fields. Unlike legacy FIX data dictionaries, Orchestra can provide a precise message layout for each event.

Here’s a simplified view of the “traded” scenario message definition:


    
        
        
        
            in.ClOrdID
        
        
        
        
        
            in.Side
        
        
            in.OrdType
        
        
            in.TimeInForce

One of the implications of scenario design is that the multitude of optional fields tends to fall away. Rather, each scenario contains just the required fields for its use case. Message layouts are thus much more deterministic than without scenarios.

How to encode conditional expressions using the Score DSL Orchestra Concepts Part 3: Condtional Expressions

Back

Orchestra Concepts Part 1: Message Structures

Concepts-Part3-Conditional-Expressions

FIXTrading Community — Fri, 02 Aug 2019 11:54:03 +0000

Orchestra Concepts Part 3:

Conditional Expressions

Conditional expressions tell when something should happen or when a rule applies. Orchestra provides a language called Score to form expressions that machines can read and evaluate.

Overview

Rules of engagement between counterparties often depend on the values in a message or external state information. One use case is conditionally required fields—whether a field is required is contingent on the value of another field. In the FIX specifications, this has traditionally been expressed in text comments, often following the phrase “required if…” Another use case for condional expressions is to select when one workflow reponse is triggered among several alternatives. Score expressions can also be used to set field values.

The problem with text narratives is that are designed for human readability. Unfortunately, they are not easily parsed by computer programs, and even to humans, the narrative is often vague or ambiguous. At least in the case of a conditionally required field, the condition is usually next to the field definition. In the case of workflow, the problem is usually worse—either the flow is not explained at all, or the documentation of the flow is disconnected from the message definitions, requiring humans to put it together in their heads.

The goal of Orchestra is to make rules of engagement machine readable, precise, and unambiguous. The rest of this tutorial gives a tutorial for how to form machine readable condtional expressions.

What’s it good for?

Receiving machine-readable expressions can be used for …

Writing messaging test cases and validations
Evaluating conditions at run-time
Populating messages from field assignment expressions

Score DSL

Score is the part of Orchestra to form expressions. It is what is called a Domain Specific Language (DSL). It was designed to be easy to learn for software developers since it is highly similar to the expression syntax of common programming languages including C, C++, C#, and Java. It should also look familiar to anyone who has written spreadsheet formulas. It was expected that there would be these two groups of users, so the syntax allows some alternatives. For example, an advanced business user can use the word “or” to combine logical expressions while a software engineer uses the symbol “||” to mean the same thing but using a form common in technical expression syntaxes. The two symbols are interchangeable in Score.

The complete syntax of Score is defined in the Orchestra specification. However, this tutorial should give enough of the gist to get you started.

Conditionally required field

Let’s say that a message definition contains this field reference:


    
        OrdType == ^Stop || OrdType == ^StopLimit
    
    
        
         Required for OrdType = "Stop" or OrdType = "Stop limit".

It is a reference to the stop price field definition, matching on its id attribute:

Before we break down the expression in the element, see if you can guess its meaning without looking at that pretty much gives it away.

(waiting…)

I hope that if you understand FIX, that meaning was apparent, but let’s analyze it carefully.

The field is by default optional in this message. That is, the field may be present in the message but is not required. However, the contained named “StopOrderRequiresStopPx” overrides the default presence by setting presence="required".

When is the rule activated? It depends on the element. The content of that element is a Score DSL expression. When the expression evalutes to true, then the rule applies, and consquently, stop price is required in the message.

Let’s examine the expression. Remember, that the symbol “||” is a synonym for the word “or”. So there are two logical expressions connected by “or”. If either of the parts is true, the whole expression evaluates to true.

There is another non-word symbol in the expression “==”, meaning equality. If the two entities on either side of an equality symbol evaluate to the same value, then the expression is true. (If you are not a programmer, I have to explain why it is a double equal sign. The reason is that equal sign actually has two meanings. One is a test of equality as used here. The other meaning is assignment, as when you assign a value to a variable in algebra. Since Score supports both tests of equality and assignment, it has to distinguish the two meanings. Therefore, like many programming languages, it uses a single equal sign for assignment and a double equal for equality.)

What is “OrdType” in the expression? It is the value of the OrdType field in the same message. The field has tag 40, but you don’t have to remeber that to write the expression because the Score implementation can cross-reference the name to its field definition.

Score lets you reference other fields in the same message, and even reference an incoming message in an expression pertaining to its response. Since “OrdType” is followed by an equality symbol, there is a test to evaluate whether the value of the OrdType field to some other value.

There is one more non-word symbol in the expression: “^”. To explain it, let’s show another piece of the Orchestra file. Notice that the type of the OrdType field above is “OrdTypeCodeSet”. That is not the name of a built-in FIX datatype, but rather is the name of a code set, shown below (simplified).

As you can see, “Stop” and “StopLimit” are names of codes in the OrdTypeCodeSet. Again, you don’t need to know the code values on the wire to write the Score expression, only the names of the codes. So, Score expressions are humanly understandable as well as machine readable.

Let’s now put together everything we’ve learned about a Score expression for a conditionally required field and see if your guess was right. The Score expression can be translated to natural language as: “if the value of the OrdType field in this message is equal to the code Stop or is equal to the code StopLimit”, then the rule states that the stop price field is required. Not only is the Score expression shorter than the English equivalent, it is unambiguous and can be parsed and evaluated by computer code. The cost to an author is learning a few special symbols, but you already know some of the inportant ones.

Conditional message responses

In the previous tutorial, we discussed workflow. A message can have one or more elements. Like a field , a can have a element containing a Score expression to tell when the response is triggered. The expression syntax is exactly the same as for a conditionally required field. If the expression evaluates to true, the response is fired. Typically, message responses involve external states as well as values in an incoming message, so we’ll defer a fuller explanation to the next tutorial.

Field assignment

A Score expression can be used to assign a field value in a response message, as shown in the example below.


    in.ClOrdID

Field 11 is ClOrdId. The element contains a Score expression to assign a value to that field in the response. Unlike the elements above, this Score expression is not a true/false predicate, but rather evaluates to a value in the datatype of ClOrdId. What is that?

Answer: datatype String. It is contrained in the definition of the response message by the attribute implMaxLength="20". In short, the outgoing ClOrdID field must be a String of no more than 20 characters.

The prefix “in.” of the assignment expression refers to the incoming message that triggered the response. To put it all together, the Score expression assigns the incoming ClOrdID value to the outgoing ClOrdID field. In other words, it echoes to the input to the output, up to 20 characters.

Actors and external states

Back

Orchestra Concepts Part 2: Workflow and Scenarios

Hands-on-Part1-Accessing-Message-Structures

FIXTrading Community — Fri, 02 Aug 2019 11:52:27 +0000

Orchestra Hands-on Tutorial Part 1:

Accessing Message Structures

An Orchestra file contains message definitions and workflows. When you receive an Orchestra file from a trading counterparty, how do you access it programmatically to discover its elements?

An Orchestra file is encoded in XML and must conform to the schema published in GitHub project FIXTradingCommunity/fix-orchestra. There are actually two XML schemas in Orchestra, in modules repository2016 and interfaces2016. The first is for message definitions and workflows, and the latter is for defining service offerings, protocol stacks, and sessions. For this tutorial, we’ll work with the repository schema, but interfaces access works the same way technologically.

Prequisites

For this tutorial, we’ll work in the Java programming language, using Maven as the build tool. So you’ll need Java 8 or later and Maven 3 for the steps below. (Of course, XML can be read and manpulated in other programming languages using appropriate tools.)

Step 1: Create a new project

Create a new simple Maven project. IDEs such as Eclipse or Intellij IDEA have menus and dialogs for this.

Code to access or update an Orchestra file is available as open source. To make it easy on yourself, let Maven retrieve a pre-built excutable for you. If you’d rather dive into detail, you can build the dependency yourself from source–see below.

Use pre-built XML bindings

XML bindings relieve you of having to deal with XML details such as elements and attributes. Rather, you just use plain old classes and objects (POJOs).

Orchestra XML bindings generated with Java Architecture for XML Binding (JAXB) are already baked into an open-source project for you to use. Add this snippet to the element of your Maven project file. (Check if there is a later version available in Maven Central repository.) When you build your project, Maven will retrieve each dependency in your project automatically.


  io.fixprotocol.orchestra
  repository2016
  1.3.0-RC4

Note: if you’re using Java 9 or later, you’ll have to add dependency for module javax.xml.bind since JAXB has been moved out of the base module.

Alternative: Generate XML bindings from source

If you wish to build the XML bindings from scratch, retrieve the source for the module described above from GitHub module repository2016. Tell your IDE to build the project, or issue this command line from the root directory of the dependency project:

mvn install

In the generate-sources phase of the build, JAXB source code is generated to access or update elements of an XML file that conforms to the Orchestra schema. Then the source code is compiled, packed into a .jar file, and stored in you local Maven repository, making it available to your new project.

Step 2: Create a class to report Orchestra messages

Let’s create a class to report the messages contained in an Orchestra file. For this to work, the class has to parse the XML file and convert XML constructs into Java objects. The class will have a method to accept an input stream for the file, and an output stream to send text to. Start with this outline:

package com.example.orchestra.sample;

import java.io.InputStream;
import java.io.PrintStream;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Unmarshaller;
import io.fixprotocol._2016.fixrepository.Repository;

public class RepositoryReporter {

  public void report(InputStream is, PrintStream os) {
    try {
      // parse the XML
      Repository repository = unmarshal(is);
      // report messages
      // TODO
    } catch (JAXBException e) {
      // print exception
      os.format("ERROR: %s%n", e.getMessage());
    }
  }

  private Repository unmarshal(InputStream is) throws JAXBException {
    final JAXBContext jaxbContext = JAXBContext.newInstance(Repository.class);
    final Unmarshaller jaxbUnmarshaller = jaxbContext.createUnmarshaller();
    return (Repository) jaxbUnmarshaller.unmarshal(is);
  }

}

To report all the messages in the Orchestra file, insert this snippet into the report() method. It accesses a list of all messages in the repository and invokes a new method to report each message.

      List messageList = repository.getMessages().getMessage();
      reportMessages(messageList, os);

The new method is:

  void reportMessages(List messageList, PrintStream os) {
    // iterate all messages in the file
    for (MessageType message: messageList) {
      os.format("Message name: %s scenario: %s MsgType: %s%n", 
          message.getName(), message.getScenario(), message.getMsgType());
      // report message members
      // TODO
    }  
  }

To list the members of a message structure, insert this snippet into the reportMessages() method. The structure of a message is mixed collection of field, component, and repeating group references.

      List members = message.getStructure().getComponentRefOrGroupRefOrFieldRef();
      reportMembers(members, os);

Add a new method to report each of the members. It has to check the type of each object in the mixed collection. To keep it simple, we’ll only report the identifiers of each member, but you can expand the code to report more attributes.

  void reportMembers(List members, PrintStream os) {
    for (Object member : members) {
      if (member instanceof FieldRefType) {
        FieldRefType fieldRef = (FieldRefType)member;
        os.format("tFieldRef id: %d scenario: %s%n", fieldRef.getId(), fieldRef.getScenario());
      } else if (member instanceof ComponentRefType) {
        ComponentRefType componentRef = (ComponentRefType)member;
        os.format("tComponentRef id: %d scenario: %s%n", componentRef.getId(), componentRef.getScenario());
      } else if (member instanceof GroupRefType) {
        GroupRefType groupRef = (GroupRefType)member;
        os.format("tGroupRef id: %d scenario: %s%n", groupRef.getId(), groupRef.getScenario());
      }
    }
  }

Your homework

Accessing fields, components, and repeating groups follows the same format, so you can write their accessors by following the style for messages.

This simple tutorial reports on each element as it is accessed. However, a real application would likely need to collect information and cross-reference elements before processing them. For example, a FieldRef member of a message is only identified by its numeric ID (field tag). To get the field name, you have to cross-refernce the Field element with the same ID.

Caution: not all XML attributes are required, so some accessors may return null. Be careful to check for nullness before attempting to use a returned object reference.

Step 3: Create a main and build the application

Create main() entry point to receive command-line arguments like so:

public class RepositoryApp {
  public static void main(String[] args) {
    if (args.length < 1) {
      usage();
    } else {
      RepositoryReporter reporter = new RepositoryReporter();
      InputStream is;
      try {
        // first argument is an Orchestra file name
        is = new FileInputStream(args[0]);
        reporter.report(is, System.out);
      } catch (FileNotFoundException e) {
        System.err.println(e.getMessage());
      }
    }
  }

  public static void usage() {
    System.err.println("Usage: RepositoryApp ");
  }
}

Tell your IDE to build the project, or issue this command line from the root directory of the project:

mvn install

Now your first Orchestra application is ready to run! Output should look something like this:

Message name: NewOrderSingle scenario: SecurityType-Future MsgType: D
    ComponentRef id: 1024 scenario: base
    FieldRef id: 11 scenario: base
    FieldRef id: 1 scenario: base
    FieldRef id: 21 scenario: SecurityType-Future
    ComponentRef id: 1003 scenario: SecurityType-Future
    FieldRef id: 54 scenario: SecurityType-Future
    FieldRef id: 60 scenario: base
    ComponentRef id: 1011 scenario: SecurityType-Future
    FieldRef id: 40 scenario: SecurityType-Future
    FieldRef id: 44 scenario: base
    FieldRef id: 59 scenario: SecurityType-Future
    ComponentRef id: 1025 scenario: base

Concepts-Part1-MessageStructures

FIXTrading Community — Fri, 02 Aug 2019 11:49:40 +0000

Orchestra Concepts Part 1:

Message Structures

An Orchestra file contains message definitions and their building blocks, one of the aspects of rules of engagement between counterparties.

Overview

Orchestra is not a communication protocol in the ususal sense, nor is it an intermediary for messages on the wire. Rather, it describes message structures and behaviors of interacting peers about their communications rules— technically, metadata rather than data. We call these expected behaviors between counterparties rules of engagement. The benefits of receiving such metadata include the ability to accurately prepare for behaviors in advance rather than discovering them at runtime, potentially leading to unexpected errors or breakdowns.

What’s it good for?

Receiving machine-readable definitions of message structures can be used for …

Developing message encoders and decoders and configuring them in a FIX engine or other communications gateway
Specifying and developing message validators and translators
Writing messaging test cases
Generating documentation for humans

At the Application Layer

Orchestra defines messages and behaviors as viewed by the application layer, not bits and bytes as viewed from lower technical protocol layers. Another way to say it is that Orchestra deals with semantics.

Orchestra was designed to work for practically any interactive message protocol, not just for FIX. Even within the family of FIX protcols, there are multiple message encodings— including the traditional TagValue format, FIXML, and binary encodings such as SBE— but they can all have the same business semantics.

A Standard File Format

The Orchestra standard has two file schemas, one for message structures and workflows of messages, and the other describes protocol stacks and session configurations. In this tutorial, we’ll cover the first one, which is historically a successor to FIX Repository. Therefore, another name for it is Repository 2016 Edition.

To ensure that an Orchestra file is interoperable, it must conform to an XML schema that is published as part of the Orchestra standard. See the repository2016 XSD files here in GitHub.

Message Structures

Orchestra is not a communications protocol, so it does not deal with instances of messages on the wire, but rather with templates for formation of messages. A message is a complete unit of data sent on the wire. It is composed of fields, components, and repeating groups. Let’s dive into these concepts and how Orchestra represents them.

Field

A field is the smallest unit of business semantics. A field always has the same meaning wherever it used. For example, the Price field is used in many message types, but it always means “price per unit of quantity”.

A field has an identifier called a tag. In FIX, tags are numeric, and FIX experts know common tags by heart, e.g. tag 44 is the identifier of the Price field. Tags are persistent— they remain the same forever for a given field and can never change or be reused. Only new field tags can be added.

How the Symbol (55) field is represented in Orchestra (simplified):


    
    
         Ticker symbol. Common, "human understood" representation of the security.

Key parts of a field definition:

The prefix “fixr:” in front of all elements refers to the Orchestra XML namespace. We’ll say no more about underlying XML technology, but it will appear in all tutorials.
The name attribute gives a humanly understood name. Typically, this name is not sent on the wire by message processors; it’s metadata to aid humans.
The id attribute, commonly known as tag, is a numeric identifier of a field. In FIX TagValue encoding, the tag is sent on the wire.
The abbrName is an abbreviated name, which is used as a shorter tag in FIXML elements since field names can be very long.
The type attribute tells the name of the datatype of a field (explained below).
The scenario attribute gives a particular useage of a field. We’ll explain scenarios when we get to message definitions.
Every message element in an Orchestra file can have humanly readable documentation as shown.

Also, message elements can be documented with pedigree, the version history of the element, not shown here for simplicity.

All XML elements are contained by their parent element in an Orchestra file.

Datatype

FIX has about 20 predefined datatypes. A datatype is a value space, that is, the range of possible values that a field can convey. In the example above, the Symbol field is of datatype String, a sequence of characters. There are other datatypes for numeric values, dates, times, and so forth.

For each datatype, mappings can be defined for multiple message encodings, such as XML, SBE, and so forth. Technically, the rules for encoding a datatype is called its lexical space. Usually, you won’t need to deal with that unless you are describing a new, non-FIX protocol, so we won’t go into that in this tutorial. You can read about it in the Orchestra specification if you need to.

Code set

There is a special kind of datatype that you probably do need to know about, however. A code set describes a finite set of valid values of a field called codes, as opposed to a continuous range of values. A code set is also known as an enumeration.

A code set has an underlying datatype for all its code values. In FIX, codes may be of String, int, or char datatypes. Lately, new code sets have all been numeric.

Here’s an example of a code set of char type that enumerates order types (simplified):

Like a field, a code set and the codes that it contains have a humanly understood name attribute, and a numeric id.

Each code has a value, the value sent on the wire. In the example, many of the code values look like numbers, but they are actually characters, as shown by code value “A”.

A code set may be used as a datatype by one or more fields. In FIX Latest, more than 20 fields share SecurityIDSourceCodeSet, but its codes only need to be defined once in an Orchestra file.

All XML elements are contained by their parent element in an Orchestra file.

Here’s how the code set above is assigned as the datatype of a field:

In a message, when OrdType field (tag 40) is sent on the wire, the character value of the field signifies one order type from the code set.

Component Block

A component block is a collection of fields that can be reused in multiple message types. The FIX standard defines many components. Instrument block is one of the most common ones; it is used in over half of FIX message types.

A simplified definition of Instrument block:


    
    
    
    
    
    
    
         The Instrument component block contains all the fields commonly used to describe a security or instrument. Typically the data elements in this component block are considered the static data of a security, data that may be commonly found in a security master database. The Instrument component block can be used to describe any asset type supported by FIX.

The elements of a component may be any combination of the following:

A is a reference to a defined within the container. The id attribute is the key for matching the reference to the field. So, matches the Symbol field as defined above. The point is, you only have to define a field once with its name, datatype, documentation, and so forth, but you can reuse that definition as many times as needed.


Components may be nested. A  is a reference to a contained component. Like a fieldRef, the id attribute is the key for matching. In this example, the component with id="1060" is called SecurityXML. All of the fields defined for that component are nested within the Instrument block.
Suffice it say that a  is a reference to a  repeating group that we’ll talk about next. Like components, repeating groups can be nested within components or other repeating groups.


We’ll explain the presence attribute in depth when we get to message, but for now just know that “optional” means that field need not be sent on every message.
All  XML elements are contained by their parent element  in an Orchestra file.
Repeating Group
A repeating group is like a component, but a repeating group can have multiple instances when sent on the wire. For example, the Parties group can carry data for multiple parties to a transaction. (Most repeating group names end in “Grp” but this one is an exception.)
When sent on the wire in TagValue encoding, the instances of a repeating group are preceeded by a counter to inform a message decoder how many instances to parse. The counter is a field of a special datatype called NumInGroup, in essence a cardinal number. The only difference between a repeating group definition and a component is the addition of  element to specify the tag of the group counter. Otherwise, it contains the same field, component and nested group reference formats.
Here’s a simplified definition of the Parties group with its NumInGroup tag.

    
    
    
    

All  XML elements are contained by their parent element  in an Orchestra file.
Message
Now that we have defined the building blocks of a message, we can define a message. Like a component definition, its structure also contains field, component and group references. However, a message has some extra parts.
First,  has an extra identifier, namely attribute msgType, the traditional FIX protocol message identifier.
Another difference between the format of a message and component is that the structure of a message resides under a  element. The reason for it is that A message can also have workflow aside from structure. That’s what the flow attribute is about, too. We’ll cover workflow in a later tutorial.
Here’s a simplified definition of an ExecutionReport message that is sent when a trade occurs:

    
        
        
        
        
            in.ClOrdID
        
        
            
            
         Specifies party information related to the submitter.
            
            
        
        
    

Message structure definitions have many optional XML attributes. One in the example, implMaxLength informs users that ClOrdID (tag 11) is limited to 20 characters.
In the message definition above, the first element in the structure,  refers to the StandardHeader component, and the last one with id="1025" refers to StandardTrailer. These are required by TagValue encoding.

All  XML elements are contained by their parent element  in an Orchestra file.
Presence
Orchestra provides these possible values of presence:

required — the field or component MUST be present.
optional — the field or component MAY be present; it may be conditionally required based on a rule.
forbidden — The field or component MUST NOT be present.
ignored — the field or component MAY be present but is not validated.
constant — the field has a constant value. (In some encodings, contants need not be sent on the wire.)

In the above example, fieldRef with id="37", the OrderID, field has attribute presence="required". This means that the rules of engagement state that in this message type, OrderID is a required field— it must be included in every instance of the message to be valid.
Frequently in FIX, fields are conditionally required, that is, it is required when some condition about the message is true. For example, StopPx field is required when the value of the OrdType field is Stop or StopLimit. Fortunately, Orchestra has syntax to express such rules — Orchestra’s Score Domain Specific Lanaguage (DSL). Score can also used to express assignment of field values. In the example, the ClOrdId field (tag 11) in the ExecutionReport is assigned the value from the incoming order message. We’ll leave the syntax of that language for an advanced tutorial, but you should get the gist of the example.
Scenarios
In FIX, message types are often overloaded for many different meanings. We call each specialization or use case of a message a scenario. For example, there may be scenarios of ExecutionReport message type for when an order is booked, when it trades, when it is rejected, or when an order is replaced.
Another reason to use scenarios is that different security types need different sets of fields to describe them. So an option order requires slightly different fields, like MaturityMonthYear (tag 200), that is not required for an equity order. An interesting thing here is that MaturityMonthYear is not directly included in the definition of NewOrderSingle (msgtype=”D”), but rather is included in the Instrument component.
To deal with different views of the Instrument block, we could define scenarios for it, one for equities, one for options, and so forth. Each is distinguished by its scenario name. An equity order only requires a Symbol or SecurityID, but an option also need MaturityMonthYear, StrikePrice, and put or call classification.

    
    
    



    
    
    
    
    
    

Another common reason to use scenarios is to have different views of code sets. Inbound orders may use one scenario of PartyRole while outbound execution reports use a more enriched version with more codes in the code set.
Next
In subsequent tutorials, we explain concepts of workflow, conditional expressions, and other details of Orchestra. The next concept tutorial is Orchestra Concepts Part 2: Workflow and Scenarios.
To learn how to access message structures programmatically, see Orchestra Hands-on Tutorial Part 1: Accessing Message Structures.

Orchestra Tutorials – FIX Trading Community

Extension Packs now with Orchestra XML File

Objective

Extension Pack Content

FAQ

Concepts-Part5-Service-Offerings-And-Session-Configurations

Service Offerings and Session Configurations

Overview

Service Offerings

Protocol Stack

Session Configuration

Concepts-Part5-Service-Offerings-And-Session-Configurations

Service Offerings and Session Configurations

Overview

Service Offerings

Protocol Stack

Session Configuration

Concepts-Part4-Actors-And-External-States

Actors and External States

Overview

State Variables

State Machines

Next

Back

Concepts-Part2-Workflow-and-Scenarios

Workflow and Scenarios

Overview

What’s it good for?

Actors and Flows

Message Workflows

Message Scenarios

Next

Back

Concepts-Part3-Conditional-Expressions

Conditional Expressions

Overview

What’s it good for?

Score DSL

Conditionally required field

Conditional message responses

Field assignment

Next

Back

Hands-on-Part1-Accessing-Message-Structures

Accessing Message Structures

Prequisites

Step 1: Create a new project

Use pre-built XML bindings

Alternative: Generate XML bindings from source

Step 2: Create a class to report Orchestra messages

Your homework

Step 3: Create a main and build the application

Copyright 2019 FIX Protocol Ltd.

Concepts-Part1-MessageStructures

Message Structures

Overview

What’s it good for?

At the Application Layer

A Standard File Format

Message Structures

Field

Datatype

Code set

Component Block

Repeating Group

Message

Presence

Scenarios

Next