Git Product home page Git Product logo

fix2json's Introduction

fix2json

NOTE

This repository is planned for public archival on 01 June 2023. As an alternative, please consider the Market Data Transcoder.

About

fix2json is a command-line utility to present FIX(TV) protocol message files as JSON or YAML (fix2yaml).

Some scenarios where fix2json might be of use are:

  • Within the ingestion pipeline of trading venue drop copy files into ElasticSearch

  • Performing command-line ETL or manipulation of FIX data, often in concert with other CLI utilities such as jq or XMLStarlet

  • As a pre-processor for inserting FIX records into JSON-conformant NoSQL repositories, such as Google BigQuery or MongoDB.

fix2json does not attempt any affirmative context validation, although syntax validation is performed as required for parsing of repeating groups. As such, users should consider ensuring the contextual validity of all FIX messages upstream. For satisfactory results, fix2json should be paired with the appropriate target FIX version data dictionary.

There is a high likelihood that the data dictionaries in dict will need to be adjusted to conform to the FIX specification of the particular message generator. In particular, the processing of repeating groups relys upon the specified dictionary to determine the end of any particular group. If the FIX message data uses custom fields that are not present in the supplied data dictionary, the dictionary really should be patched accordingly. See the files dict/FIX50SP2.CME.xml and dict/FIX42-bloomberg-step.xml for examples of vendor-specific tags retroactively patched to a stock QuickFIX XML data dictionary.

Install

npm install -g fix2json

Examples

$ fix2json
Usage: fix2json [-p] <data dictionary xml file> [<path to FIX message file>]

$ fix2json -p dict/FIX50SP2.CME.xml XCME_MD_GE_FUT_20160315.gz 
{
  "ApplVerID": "FIX50SP2",
  "BodyLength": 131,
  "MsgType": "SECURITYSTATUS",
  "SenderCompID": "CME",
  "MsgSeqNum": 9750889,
  "SendingTime": "20160314214500018844412",
  "TransactTime": "20160314214500001407698",
  "TradeDate": "20160315",
  "MatchEventIndicator": "00000000",
  "SecurityGroup": "GE",
  "SecurityTradingStatus": "PRE OPEN",
  "HaltReasonInt": "NEWS DISSEMINATION",
  "SecurityTradingEvent": "CHANGE OF TRADING SESSION",
  "CheckSum": "083"
}
{
  "ApplVerID": "FIX50SP2",
  "BodyLength": 182,
  "MsgType": "MARKETDATAINCREMENTALREFRESH",
  "SenderCompID": "CME",
  "TradeDate": "20160315",
  "MsgSeqNum": 9751026,
  "SendingTime": "20160314214500474778917",
  "TransactTime": "20160314214500474587143",
  "MatchEventIndicator": "00000100",
  "NoMDEntries": 1,
  "MDEntries": [
    {
      "MDUpdateAction": "CHANGE",
      "MDEntryType": "OFFER",
      "SecurityID": "801250",
      "Symbol": "GEH7",
      "RptSeq": 1501314,
      "MDEntryPx": 9894.5,
      "MDEntrySize": 665,
      "NumberOfOrders": 9,
      "MDPriceLevel": 1
    }
  ],
  "CheckSum": "226"
}
{
  "ApplVerID": "FIX50SP2",
  "BodyLength": 183,
  "MsgType": "MARKETDATAINCREMENTALREFRESH",
  "SenderCompID": "CME",
  "TradeDate": "20160315",
  "MsgSeqNum": 9751027,
  "SendingTime": "20160314214500474803975",
  "TransactTime": "20160314214500474666839",
  "MatchEventIndicator": "00000100",
  "NoMDEntries": 1,
  "MDEntries": [
    {
      "MDUpdateAction": "CHANGE",
      "MDEntryType": "OFFER",
      "SecurityID": "801075",
      "Symbol": "GEU6",
      "RptSeq": 1151986,
      "MDEntryPx": 9911.5,
      "MDEntrySize": 817,
      "NumberOfOrders": 13,
      "MDPriceLevel": 1
    }
  ],
  "CheckSum": "040"
}
{
  "ApplVerID": "FIX50SP2",
  "BodyLength": 183,
  "MsgType": "MARKETDATAINCREMENTALREFRESH",
  "SenderCompID": "CME",
  "TradeDate": "20160315",
  "MsgSeqNum": 9751028,
  "SendingTime": "20160314214500475466516",
  "TransactTime": "20160314214500475387568",
  "MatchEventIndicator": "00000100",
  "NoMDEntries": 1,
  "MDEntries": [
    {
      "MDUpdateAction": "CHANGE",
      "MDEntryType": "OFFER",
      "SecurityID": "800971",
      "Symbol": "GEM6",
      "RptSeq": 940052,
      "MDEntryPx": 9920.5,
      "MDEntrySize": 3438,
      "NumberOfOrders": 18,
      "MDPriceLevel": 1
    }
  ],
  "CheckSum": "030"
}
...

$ fix2json dict/FIX42.xml fixmsg_11212014.txt | mongoimport --drop --collection FIX
connected to: 127.0.0.1
2015-08-20T15:19:21.465-0400 dropping: test.FIX
2015-08-20T15:19:24.013-0400             15600	5200/second
2015-08-20T15:19:27.000-0400 			34600	5766/second
2015-08-20T15:19:30.005-0400 			53400	5933/second
2015-08-20T15:19:33.009-0400 			72900	6075/second
2015-08-20T15:19:36.005-0400 			88500	5900/second
2015-08-20T15:19:39.010-0400 			108200	6011/second
2015-08-20T15:19:42.005-0400 			127300	6061/second
2015-08-20T15:19:45.001-0400 			146800	6116/second
2015-08-20T15:19:48.001-0400 			166000	6148/second
2015-08-20T15:19:51.429-0400 			183600	6120/second
2015-08-20T15:19:54.010-0400 			200200	6066/second
2015-08-20T15:19:55.980-0400 check 9 212993
2015-08-20T15:19:55.981-0400 imported 212993 objects


$ head -1 testfiles/100FIX42.dat | fix2json -p dict/FIX42.xml 
{
    "8201": "1",
    "Account": "909646300",
    "BeginString": "FIX.4.2",
    "BodyLength": "0",
    "CheckSum": "0",
    "ClOrdID": "0c968e69-c3ff-4f9f-bc66-9e5ebccd9807",
    "Currency": "USD",
    "IDSource": "CUSIP",
    "MsgSeqNum": "0",
    "MsgType": "ORDER SINGLE",
    "OrderQty": "75700",
    "OrdType": "MARKET",
    "OrigClOrdID": "e0568b5c-8bb1-41f0-97bf-5eed32828c24",
    "SecurityID": "464288430",
    "SenderCompID": "830199",
    "SendingTime": "20150406-12:17:27",
    "Side": "BUY",
    "Symbol": "SJM",
    "TargetCompID": "AZKJ",
    "TargetSubID": "3620",
    "TimeInForce": "DAY",
    "TransactTime": "20150406-12:17:27",
    "SecurityExchange": "P"
}

$ head -3 MDFF_CME_20130714-20130715_7819_0 | fix2yaml -p dict/FIX50SP2.CME.xml
ApplVerID: FIX50SP2
BodyLength: 274
MsgType: MARKETDATAINCREMENTALREFRESH
SenderCompID: CME
MsgSeqNum: 2918
SendingTime: '20130714212647805'
TradeDate: '20130715'
NoMDEntries: 3
MDEntries:
    -
        MDUpdateAction: NEW
        SecurityIDSource: 'EXCHANGE SYMBOL'
        SecurityID: '2975'
        RptSeq: 6
        SecurityDesc: 6EU4
        MDEntryType: 'SIMULATED SELL PRICE'
        MDEntryPx: 13157
        MDEntryTime: '212647000'
    -
        MDUpdateAction: NEW
        SecurityIDSource: 'EXCHANGE SYMBOL'
        SecurityID: '2975'
        RptSeq: 7
        SecurityDesc: 6EU4
        MDEntryType: 'SIMULATED BUY PRICE'
        MDEntryPx: 13069
        MDEntryTime: '212647000'
    -
        MDUpdateAction: NEW
        SecurityIDSource: 'EXCHANGE SYMBOL'
        SecurityID: '2975'
        RptSeq: 8
        SecurityDesc: 6EU4
        MDEntryType: OFFER
        MDEntryPx: 13157
        MDEntrySize: 10
        MDEntryTime: '212647000'
        TradingSessionID: '0'
        NumberOfOrders: 1
        MDPriceLevel: 1
CheckSum: '092'

ApplVerID: FIX50SP2
BodyLength: 277
MsgType: MARKETDATAINCREMENTALREFRESH
SenderCompID: CME
MsgSeqNum: 2916
SendingTime: '20130714212639044'
TradeDate: '20130715'
NoMDEntries: 3
MDEntries:
    -
        MDUpdateAction: NEW
        SecurityIDSource: 'EXCHANGE SYMBOL'
        SecurityID: '22195'
        RptSeq: 6
        SecurityDesc: 6EZ4
        MDEntryType: 'SIMULATED SELL PRICE'
        MDEntryPx: 13171
        MDEntryTime: '212639000'
    -
        MDUpdateAction: NEW
        SecurityIDSource: 'EXCHANGE SYMBOL'
        SecurityID: '22195'
        RptSeq: 7
        SecurityDesc: 6EZ4
        MDEntryType: 'SIMULATED BUY PRICE'
        MDEntryPx: 13083
        MDEntryTime: '212639000'
    -
        MDUpdateAction: NEW
        SecurityIDSource: 'EXCHANGE SYMBOL'
        SecurityID: '22195'
        RptSeq: 8
        SecurityDesc: 6EZ4
        MDEntryType: OFFER
        MDEntryPx: 13171
        MDEntrySize: 10
        MDEntryTime: '212639000'
        TradingSessionID: '0'
        NumberOfOrders: 1
        MDPriceLevel: 1
CheckSum: '227'

ApplVerID: FIX50SP2
BodyLength: 134
MsgType: MARKETDATAINCREMENTALREFRESH
SenderCompID: CME
MsgSeqNum: 1371
SendingTime: '20130714180142945'
TradeDate: '20130715'
NoMDEntries: 1
MDEntries:
    -
        MDUpdateAction: NEW
        SecurityIDSource: 'EXCHANGE SYMBOL'
        SecurityID: '9296'
        SettlDate: '20130712'
        RptSeq: 1
        SecurityDesc: 6EH4
        MDEntryType: 'SETTLEMENT PRICE'
        MDEntryPx: 13077
        MDEntryTime: '180142000'
CheckSum: '107'

# If streaming gzipped files over the network, let gunzip do the uncompression

$ curl -s ftp://ftp.cmegroup.com/sbe_sample_data/historical_data/XCBT_MD_ZS_FUT_20160315.gz| gunzip - | fix2yaml -p dict/FIX50SP2.CME.xml
ApplVerID: FIX50SP2
BodyLength: 278
MsgType: MARKETDATAINCREMENTALREFRESH
SenderCompID: CME
TradeDate: '20160315'
MsgSeqNum: 3254145
SendingTime: '20160314214500015068553'
TransactTime: '20160314214500001107404'
MatchEventIndicator: '00001000'
NoMDEntries: 3
MDEntries:
    -
        MDUpdateAction: NEW
        MDEntryType: 'OPENING PRICE'
        SecurityID: '827723'
        Symbol: ZSK6
        RptSeq: 717363
        MDEntryPx: 894.25
        OpenCloseSettlFlag: 'THEORETICAL PRICE VALUE'
    -
        MDUpdateAction: NEW
        MDEntryType: 'OPENING PRICE'
        SecurityID: '401316'
        Symbol: ZSN6
        RptSeq: 621346
        MDEntryPx: 898
        OpenCloseSettlFlag: 'THEORETICAL PRICE VALUE'
    -
        MDUpdateAction: NEW
        MDEntryType: 'OPENING PRICE'
        SecurityID: '253214'
        Symbol: ZSX6
        RptSeq: 617653
        MDEntryPx: 904.5
        OpenCloseSettlFlag: 'THEORETICAL PRICE VALUE'
CheckSum: '098'

ApplVerID: FIX50SP2
BodyLength: 131
MsgType: SECURITYSTATUS
SenderCompID: CME
MsgSeqNum: 3254151
SendingTime: '20160314214500017078341'
TransactTime: '20160314214500001107404'
TradeDate: '20160315'
MatchEventIndicator: '00000000'
SecurityGroup: ZS
SecurityTradingStatus: 'PRE OPEN'
HaltReasonInt: 'NEWS DISSEMINATION'
SecurityTradingEvent: 'CHANGE OF TRADING SESSION'
CheckSum: '072'

ApplVerID: FIX50SP2
BodyLength: 180
MsgType: MARKETDATAINCREMENTALREFRESH
SenderCompID: CME
TradeDate: '20160315'
MsgSeqNum: 3254163
SendingTime: '20160314214500086778208'
TransactTime: '20160314214500086645710'
MatchEventIndicator: '00000100'
NoMDEntries: 1
MDEntries:
    -
        MDUpdateAction: CHANGE
        MDEntryType: OFFER
        SecurityID: '827723'
        Symbol: ZSK6
        RptSeq: 717364
        MDEntryPx: 895.75
        MDEntrySize: 12
        NumberOfOrders: 7
        MDPriceLevel: 5
CheckSum: '156'

ApplVerID: FIX50SP2
BodyLength: 178
MsgType: MARKETDATAINCREMENTALREFRESH
SenderCompID: CME
TradeDate: '20160315'
MsgSeqNum: 3254164
SendingTime: '20160314214500088853511'
TransactTime: '20160314214500088741888'
MatchEventIndicator: '00000100'
NoMDEntries: 1
MDEntries:
    -
        MDUpdateAction: NEW
        MDEntryType: OFFER
        SecurityID: '401316'
        Symbol: ZSN6
        RptSeq: 621347
        MDEntryPx: 902
        MDEntrySize: 1
        NumberOfOrders: 1
        MDPriceLevel: 2
CheckSum: '025'

ApplVerID: FIX50SP2
BodyLength: 179
MsgType: MARKETDATAINCREMENTALREFRESH
SenderCompID: CME
TradeDate: '20160315'
MsgSeqNum: 3254165
SendingTime: '20160314214500505311711'
TransactTime: '20160314214500505220176'
MatchEventIndicator: '00000100'
NoMDEntries: 1
MDEntries:
    -
        MDUpdateAction: CHANGE
        MDEntryType: BID
        SecurityID: '253214'
        Symbol: ZSX6
        RptSeq: 617654
        MDEntryPx: 898
        MDEntrySize: 21
        NumberOfOrders: 2
        MDPriceLevel: 9
CheckSum: '078'

ApplVerID: FIX50SP2
BodyLength: 178
MsgType: MARKETDATAINCREMENTALREFRESH
SenderCompID: CME
TradeDate: '20160315'
MsgSeqNum: 3254166
SendingTime: '20160314214500524996190'
TransactTime: '20160314214500524917388'
MatchEventIndicator: '00000100'
NoMDEntries: 1
MDEntries:
    -
        MDUpdateAction: NEW
        MDEntryType: OFFER
        SecurityID: '122779'
        Symbol: ZSQ6
        RptSeq: 553424
...

Notes

  • Gzipped files being read from the filesystem and having a .gz extension will be uncompressed automatically

  • Data dictionary discrepancies will throw a monkeywrench into the processing of repeating groups. This is because fix2json refers back to the data dictionary during repeating group processing to determine when any individual group tags ends. However, if a group member contains a tag does not reside in the specified dictionary, fix2json will truncate the group prematurely, triggering rage in its victims. In this case, the best approach is to reconcile the data dictionaries employed and adjusted the dictionary accordingly. Unrecognized individual tags (not part of a repeating group) will simply be represented as the FIX tag number and its corresponding value, indicating a data dictionary mismatch less destructively.

  • fix2json makes few semantic judgements about the FIX data being processed and blindly applies the specified data dictionary to the input FIX file. Feedback from the community on an appropriate level of validation is welcomed.

  • fix2json uses streams instead of loading up the entire source file in one go. This means that you can process 2G+ files and receive output almost immediately. What this also means is that fix2json does not emit a canonical JSON array as it's output. Individual JSON objects will be output that correspond to the individual messages in the source file. For this reason, pretty printing is disabled by default. Use -p as the first argument for pretty printed output JSON.

  • fix2json will replace underscores ('_') with spaces (' ') for all mnemonic tag descriptions found in the input data dictionary. Hence, a description for EventType will be in the data dictionary as "LAST_ELIGIBLE_TRADE_DATE", but will be interpreted by fix2json as "LAST ELIGIBLE TRADE DATE"

See Also

License

MIT. See license text in LICENSE.

Copyrights and Names

Copyright © FIS 2015-2016. Licensed under the MIT license.

fix2json's People

Contributors

salsferrazza avatar shatteringlass avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

alisheikh chishiuanlu eginis fjuza cornfeedhobo fsnlarson salsferrazza lsac algoskynet dendisuhubdy 5l1v3r1 hbnworkstation rajendra1 kabbalahoracle compqabbalah shatteringlass vl-impact johngreenan

fix2json's Issues

/usr/bin/env: ‘node’: No such file or directory

After the following two steps to install fix2json as per the description
sudo apt install npm
sudo npm install -g fix2json

I ran the same command:
fix2json -p dict/FIX50SP2.CME.xml XCME_MD_GE_FUT_20160315.gz
and I received error of: /usr/bin/env: ‘node’: No such file or directory

It would be appreciated if you can point out what is missing here?

Thank you.

CME Instruments File

Has anyone tried to parse the CME instruments file? I'm trying to get them to dump using the repository dictionary and I'm getting errors.

File is located here - ftp://ftp.cmegroup.com/SBEFix/Production/secdef.dat.gz

Support for compress files

I have been working with fix2json for a while thank you guys for this package. Just want to suggest that support for compress files would be great as FIX files are usually pretty large. What I have been doing is redirecting the stdout from gunzip to fix2json.

$ gunzip -c XCME_201301.gz | ./fix2json.js dict/FIX50SP2.CME.xml > XCME_201301.json

Dictionaries update

Hi guys, thank you for developing this tool it has been really helpful on my research. I am having a issue with a new data set from the CME that has custom tags. How can I add these tags to the custom dictionary? Here is part of the FIX msg and the same decoded msg:

CME Fix message

1128=99=43135=d49=CME34=32252=2013071416010001415=USD22=848=11147355=ES107=ESU4200=201409202=0207=XCME461=FFIXSX462=5562=1731=1827=2864=2865=5866=201306211145=133000000865=7866=201409191145=133000000870=3871=24872=1871=24872=4871=24872=14947=USD969=25996=IPNT1140=20001141=11022=GBX264=101142=F1143=6001146=12.51147=501148=1562501149=1729501150=1643501151=ES1180=75796=201307129787=0.019850=010=071
1128=99=51635=d49=CME34=32352=2013071416010001515=USD22=848=53312355=ES107=ESZ3-ESH4200=201312202=0207=XCME461=FMIXSX462=5555=2600=[N/A]602=28112603=8623=1624=2600=[N/A]602=382206603=8623=1624=1562=1731=1762=EQ827=2864=2865=5866=201212211145=143000000865=7866=201312201145=143000000870=4871=24872=1871=24872=4871=24872=11871=24872=14947=USD969=5996=CTRCT1140=50001141=11022=GBX264=101142=F1143=501144=01146=01147=01150=-6401151=ES1180=75796=201307129787=0.019850=010=167
1128=99=51535=d49=CME34=32452=2013071416010001615=USD22=848=11112955=ES107=ESM4-ESU4200=201406202=0207=XCME461=FMIXSX462=5555=2600=[N/A]602=8347603=8623=1624=2600=[N/A]602=111473603=8623=1624=1562=1731=1762=EQ827=2864=2865=5866=201306211145=133000000865=7866=201406201145=133000000870=4871=24872=1871=24872=4871=24872=11871=24872=14947=USD969=5996=CTRCT1140=50001141=11022=GBX264=101142=F1143=501144=01146=01147=01150=-7001151=ES1180=75796=201307129787=0.019850=010=130
1128=99=51535=d49=CME34=32552=2013071416010001715=USD22=848=12033555=ES107=ESH4-ESM4200=201403202=0207=XCME461=FMIXSX462=5555=2600=[N/A]602=382206603=8623=1624=2600=[N/A]602=8347603=8623=1624=1562=1731=1762=EQ827=2864=2865=5866=201303151145=133000000865=7866=201403211145=133000000870=4871=24872=1871=24872=4871=24872=11871=24872=14947=USD969=5996=CTRCT1140=50001141=11022=GBX264=101142=F1143=501144=01146=01147=01150=-7001151=ES1180=75796=201307129787=0.019850=010=117

Decoded using fix2json

{
    "5796": "20130712",
    "9787": "0.01",
    "9850": "0",
    "ApplVerID": "FIX50SP2",
    "BodyLength": "431",
    "MsgType": "SECURITYDEFINITION",
    "SenderCompID": "CME",
    "MsgSeqNum": "322",
    "SendingTime": "20130714160100014",
    "Currency": "USD",
    "SecurityIDSource": "EXCHANGE SYMBOL",
    "SecurityID": "111473",
    "Symbol": "ES",
    "SecurityDesc": "ESU4",
    "MaturityMonthYear": "201409",
    "StrikePrice": "0",
    "SecurityExchange": "XCME",
    "CFICode": "FFIXSX",
    "UnderlyingProduct": "5",
    "MinTradeVol": "1",
    "SettlPriceType": "FINAL",
    "ExpirationCycle": "TRADING ELIGIBILITY EXPIRATION SPECIFIED IN THE DATE AND TIME FIELDS EVENTDATE",
    "NoEvents": "2",
    "Events": [
        {
            "EventType": "ACTIVATION",
            "EventDate": "20130621",
            "EventTime": "133000000"
        },
        {
            "EventType": "LAST ELIGIBLE TRADE DATE",
            "EventDate": "20140919",
            "EventTime": "133000000",
            "NoInstrAttrib": "3",
            "InstrAttrib": [
                {
                    "InstrAttribType": "TRADE TYPE ELIGIBILITY DETAILS FOR SECURITY",
                    "InstrAttribValue": "1"
                },
                {
                    "InstrAttribType": "TRADE TYPE ELIGIBILITY DETAILS FOR SECURITY",
                    "InstrAttribValue": "4"
                },
                {
                    "InstrAttribType": "TRADE TYPE ELIGIBILITY DETAILS FOR SECURITY",
                    "InstrAttribValue": "14"
                }
            ]
        }
    ],
    "UnitOfMeasure": "IPNT",
    "MaxTradeVol": "2000",
    "NoMDFeedTypes": "1",
    "MDFeedTypes": [
        {
            "MDFeedType": "GBX",
            "MarketDepth": "10"
        }
    ],
    "MaxPriceVariation": "600",
    "MinPriceIncrementAmount": "12.5",
    "UnitOfMeasureQty": "50",
    "LowLimitPrice": "156250",
    "HighLimitPrice": "172950",
    "TradingReferencePrice": "164350",
    "SecurityGroup": "ES",
    "ApplID": "7",
    "CheckSum": "071",
    "StrikeCurrency": "USD",
    "MinPriceIncrement": "25",
    "MatchAlgorithm": "F"
}

Add extension pack support for dictionaries

Figure out issues with Sebastian Tomac's converter from 2011 or apply extension pack zipfiles via CLI or filesystem convention. This would obviate XML merging of manually patched QuickFIX dictionaries. Or fix2json can stop using qfx dictionaries and just use native repository definitions and extension packs, although will have to make sure FIX licensing is compatible with distribution of those assets alongside fix2json.

Component fields in 5.x dictionaries not accommodated

When using 5.x FIX dictionaries (e.g. processing CME Datamine historicals), component field definitions under repeating groups are ignored and thus groups containing sub-components (e.g. NoLegs) in their dictionary schemae are truncated and yield a semantically incorrect interpretation of the inbound FIX message.

{
    "ApplVerID": "FIX50SP2",
    "BodyLength": "565",
    "MsgType": "SECURITYDEFINITION",
    "SenderCompID": "CME",
    "MsgSeqNum": "9656",
    "SendingTime": "20130922161002713",
    "Currency": "USD",
    "SecurityIDSource": "EXCHANGE SYMBOL",
    "SecurityID": "96026",
    "Symbol": "PW",
    "SecurityDesc": "H2LU3-H2LV3",
    "MaturityMonthYear": "201309",
    "StrikePrice": "0",
    "SecurityExchange": "XNYM",
    "CFICode": "FMMXSX",
    "UnderlyingProduct": "16",
    "NoLegs": "2",
    "Legs": [
        {
            "LegSymbol": "[N/A]"
        }
    ],
    "LegSecurityIDSource": "8",
    "LegRatioQty": "1",
    "LegSide": "2",
    "LegSymbol": "[N/A]",
    "LegSecurityID": "305184",
    "MinTradeVol": "1",
    "SettlPriceType": "FINAL",
    "SecuritySubType": "SP",
...

The dictionary parser does not go within the 'InstrumentLeg' component below to catalog its fields for inclusion within group field candidacy (necessary for the algorithm to determine when it has reached the end of a repeating group in the source FIX).

...
  </component>
  <component name='TrdInstrmtLegGrp'>
   <group name='NoLegs' required='N'>
    <component name='InstrumentLeg' required='N' />
    <field name='LegQty' required='N' />
    <field name='LegSwapType' required='N' />
    <field name='LegReportID' required='N' />
    <field name='LegNumber' required='N' />
    <component name='LegStipulations' required='N' />
    <field name='LegPositionEffect' required='N' />
    <field name='LegCoveredOrUncovered' required='N' />
...

The dictionary code could use a bit of refactoring out of the main source file and into its own module. It might make sense to approach the resolution to this issue with that in mind. Solution likely requires a recursive function in processing the DOM nodes since components can have components as components.

Validate FIX repeating groups handling

fix2json doesn't have any special handling of FIX tagval repeating groups; the proper handling would obviously be nesting the objects within the JSON interpretation as represented by the incoming FIX message file.

Need to source (or synthesize) example message file specimens inclusive of repeating groups.

http://fixwiki.org/fixwiki/NoTradingSessions is an example of a repeating field with "No" interpreted as an abbreviation for "Number".

Return integer data type

I was wondering if its possible that during the decoding process from FIX to json, instate of returning strings " " as output ei. ( "MDEntryPx" : "167250" ) it returns an integer data type ei. ( "MDEntryPx" : 167250 ) currently I am using a mongo script for this but would love if this can be achieve during the decoding process.

XML merge for data dictionaries

Add support for additive specification of dictionary adjustments.

e.g. with XML merge, such that the specification different dialects reflects simply the deltas off the stock spec.

Specify data dictionary name in lieu of full path to XML

Instead of requiring the full path of the data dictionary to be specified at runtime, allow user to simply specify the dictionary name (such as FIX44 or FIX50SP2, then manufacture the full path of dictionary file internally before fetching. If the parameter includes the characters / or ., assume that the input is indeed a path and read that instead, vs. manufacturing the path.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.