Welcome to NxCore
Welcome to the NxCore API. (NxCore is pronounced "N" Core or "'Encore").
The NxCore API is the fastest possible access to the Ultra-low latency NxCore Financial Data Stream (NxCore Feed). There are 3 basic components that were installed on your computer.
1. NxCoreAccess
NxCoreAccess is a program that maintains a connection to the NxCore Feed Servers and is responsible for bringing the NxCore Feed into your computer. It also can download historic NxCore Feed data (one or more days older) in the background. You can run it as an application or as a service.
NxCore Access is the first program you will see after running NxInstall. After the login prompt, it will display a chart showing the data flowing from NxCore Servers to your computer in one second intervals. There are 4 main views which are accessed from tiny buttons labelled C (chart), T (Tape files), L (Event Log), and what looks like the equals sign (compact mode). There is a menu which appears when you click on the "N" in NxCoreAccess on the Title Bar -- the "N" is in the top left corner where the system menu is commonly found (it is the system menu).
NxCoreAccess was designed to run all the time -- 24 hours a day, 7 days a week and requires very little input from you. When you click on the upper right "x", which typically closes an application, NxCoreAccess disappears from your desktop and is visible only in the tray.
NxCore Tape
Each day, NxCoreAccess produces one file on your computer, an NxCore tape, which contains all the quotes and trades (and everything else) that was sent in the NxCore Feed for that day, beginning at exactly 12:00am and ending at 12:00pm Eastern Time. These tape files are stored in the [X]:\NxCoreData directories for the one or more drives (double click the 2nd cell from the right on the top bar in NxCoreAccess to configure drives). The naming convention for the NxCore Tapes is: YYYYMMDD.ABCDE.nxc
If the connection between NxCoreAccess and the NxCore Servers is broken, due to power outage, network outage, or a hardware failure on your machine, NxCoreAccess will always reconnect so that the NxCore Tape File(s) will not have any missing or duplicate or unreadable data. There are several layers of error checking that NxCoreAccess uses when identifying the proper sequencing for each NxCore Tape. During development, it was routine to never close the NxCore Tape files properly -- simply shutting down the application from the debugger, or from power outages (no UPS), or even intentional abuse to the file. In all cases, the NxCore Tape files were automatically repaired and verified to be identical to the original NxCore Tape files as sent from the NxCore Servers. This means you can let NxCoreAccess worry about the details in delivering error-free and complete trade and quote streams
The contents of a NxCore Tape depends on exchange permissions. Every completed tape of the same name is exactly identical to any other completed tape of the same name, worldwide.
NxCoreAccess connects to port 55555 (five 5's) on an NxCore Server. As of January 2009, there are three NxCore Servers: nxc1.nxcore.net, nxc2.nxcore.net, nxc3.nxcore.net
2. NxCoreAPI.dll
The NxCoreAPI.dll is a Dynamic Link Library that your application dynamically links with, to access the NxCore Feed. Don't worry if you have not dynamically linked to dll's before, it is a simple procedure and there are plently of working code examples so you can just cut and paste a few lines of code. The methods you use from NxCoreAPI.dll to process the NxCore Feed are collectively referred to as the NxCore API (application program interface).
There are less than 20 functions in NxCoreAPI.dll, and only one function is required -- NxCoreProcessTape which starts the data streaming into your application. One of the parameters you pass NxCoreProcessTape is the address of one of your functions (the callback function) which will be called for each trade or quote update. Your callback function is passed two formal parameters which are pointers to fixed data structures containing the quote and trade data. Your job as an application programmer is copy the data members that you are interested in.
Note:
NxCoreAPI.dll connects to NxCoreAccess's internal memory buffers to process the real-time data stream. It can also process any NxCore tape file without the need for NxCore Access.
When NxCoreAPI.dll is connected to NxCoreAccess and processing the real-time stream, you can exit NxCore Access, and the NxCoreAPI.dll will patiently wait for NxCoreAccess to return at the exact last message it processed -- so that when NxCoreAccess is started again, your callback function will receive the exact same next record as if NxCore Access was never stopped. This means you can upgrade to new versions of NxCore Access without having to restart or resynchronize in any way with the NxCore Feed. NxCore Tapes are deterministic: you can replay an NxCore Tape any time, and your call back function will always be called the exact same number of times with the exact same information. NxCore tapes with the same date and permission set will be identical on any two machines in the world. This fact will save you considerable time tracking down and reproducing problems even if your development team is remote.
You can run more than one instance of NxCoreAPI.dll simultaneously -- from the same process, or from different processes. Each NxCoreAPI instance can process the real-time NxCore Tape or a previously saved NxCore Tape. Each instance of NxCoreAPI is safely isolated from other instances so that there are practically zero contention issues. Typically, 2 instances of NxCoreAPI.dll can process 2 streams using the full 100% of CPU on a dual-processor machine, and 4 instances can simultaneously process 4 streams using the full 100% of a quad-processor machine because there is zero contention between the NxCoreAPI.dll instances. Each instance has it's own private memory pool which means if a process or thread has a bug that corrupts it's memory structures, the other instances will continue running unharmed.
NxCore Access can simultaneously service multiple threads and multiple processes connected to it's real-time memory buffers. It is structured so that if one thread or process misbehaves, or is stopped in a debugger for a long time, the other processes and threads will continue to run uninterrupted.
3. NxCoreAPI Sample applications and NxCoreAPIViewer.exe.
There are several sample applications with source code that illustrate basic concepts. You will find these applications typically in your Program Files directory which is typically:
C:\Program Files\Nanex\NxCoreAPI\Doc
Tip: An easy way to navigate to NxCore files is to click on the Diagnostics toolbar button within NxCoreAccess, and select one of the menu items that begins with the text Explore. For example, Explore Program Directory will open Window's Explorer with the path set to the NxCore programs.
Once you are receiving data in NxCore Access, click the first toolbar button labelled Run Viewer to launch the program NxCoreAPIViewer which is a collection of interactive programs that display the NxCore data fields available to your callback function. The main display of NxCoreAPIViewer presents a tree view of the actual "C" structure members of the parameters passed to your callback function.
After viewing the data in NxCoreAPIViewer, I recommend opening the Visual C++ project file for SampleOne. SampleOne is a very simple program including "C" source that illustrates the minimal elements an application needs to process the NxCore Feed. Stripped of frills, it makes the basics easier to see and understand and offers a good starting point.
Ignore the Category message types in the beginning until you master the basic quote (ExgQuotes and MMQuotes) and Trade updates. You do not need any information the Category messages for processing Quote and Trade messages.
Miscellaneous Notes
Warning:
NxCore sample code updates can and will overwrite or even delete files in directories beneath the Nanex root: C:\Program Files\Nanex. Therefore, it is strongly recommended that you always make a copy of whatever files you are working with into a seperate directory. You can always install older versions, but that won't restore any files or changes you may have added to C:\Program Files\Nanex and below.
NxCoreAccess, for example, copies NxCoreAPIViewer.exe and NxCoreAPI.dll to it's working directory when you select "Open NxCoreAPI Viewer" from the system menu (you can see the working directory path in the LogView window of NxCoreAccess). This way, there are no collisions when new versions of NxCoreAPI components are available.
NxCore Versioning
NxCore uses a common Major.Minor.Build versioning system. The build date and time are also included with the version number. NxCoreAPI will archive the current version in a seperate directory whenever the Major or Minor version number changes. This allows you to easily compare changes to any include files or text files, and also allows you to run new versions alongside of older version giving you confidence and control over the upgrade schedule.
For example. When the current version of NxCoreAPI changed from 2.2 to 2.3, a new versioned subdirectory was created:
C:\Program Files\Nanex\NxCoreAPI\Version\2.3
The new 2.3 files were copied into this new version subdirectory directly and also replaced any files in the main NxCoreAPI directory
C:\Program Files\Nanex\NxCoreAPI\
The 2.2 version files would still be available in the version subdirectory that was created when it was installed:
C:\Program Files\Nanex\NxCoreAPI\Version\2.2
So the current version is always found in 2 places: the main NxCoreAPI directory, and it's version subdirectory. Use the main NxCoreAPI directory if you always want to process the latest version, and use a version subdirectory to control which version of NxCoreAPI.dll you use.
Note: There have been only 3 minor versions from 1st quarter 2005 through 2nd quarter 2006. NxCoreAPI for the most part should remain unchanged and consistent over the years to come. Changes are almost exclusively found in new category messages and fields.
NxCore Philosophy
The underlying philosophy behind NxCore is
"to be as simple as possible, delivering the richest data possible, using the smallest memory/CPU footprint possible, but without in any way compromising the basic fundamental task of delivering quote and trade updates as fast as the state of the art allows."
Each data field in the memory blocks passed to your callback function was selected based on this philosophy. You will discover many fields available with each quote/trade update in NxCore Feed that you will not find in any other feed. An understandable initial reaction is that this extra information impacts update performance and NxCoreAPI would be even faster if it had to update less. But in fact, this is not the case: the organization of the data and each fields impact on speed (latency) was exhaustively scrutinized before making the selection for inclusion. Since these fields are already available with each update, just sitting in memory for immediate use, you will find there is much less work that you have do to (and much less CPU time you need to consume), and the overall effect with be significant faster record processing. One simple example to illustrate this:
A common initial task in any feed processing application is mapping the feed symbol strings to your data structures. A task that if done improperly, will consume a lot of CPU time (imagine a string compare for each update -- at 150,000 updates/second). Each quote/trade update in NxCoreAPI offers several fields in memory (4 actually) to assist various methods of FAST symbol mapping. Two of these fields are unique slots in each symbol space that you can fill with any 32-bit value you wish, and will be preserved automatically and appear each time that symbol updates. One incredibly simple, yet effective symbol mapping method would be to store your data stucture for each symbol in one of those slots which would reduce your indexing time cost to nil.
NxCore Processing Benchmark
The benchmark Quote/Trade processing rate for a dual 1.4 GHz Pentium class machine with 64 megabytes of free RAM is 800,000 to 1 million quote/trade updates per second. Processing speeds are approximately linear relative to CPU speed: a 2.8 GHz Pentium will process approximately twice as fast as a 1.4 GHz Pentium, though the memory performance starts to become more important at faster speeds -- that is, a system with faster memory (lower memory access latencies), will perform noticeably better. Nonetheless, with peak active real-time update rates today (2006) of 140,000 updates per second, you will find ample CPU available for your application when processing the NxCore Feed.
NxCore Server Components
NxCore Feed Processor
The NxCore Feed Processor resides on the NxCore Network that reads in real-time data streams, perform analysis, and then outputs the NxCore Feed for distribution to the NxCore Feed Servers.
NxCore Feed Server
The NxCore Feed Servers distribute the NxCore Feed to NxCoreAccess applications running on customer machines