Intro
Hydra is a program for downloading and storage of market data. This article will explain how to download the history from
Google Finance,
Gain Capital broker and
IQFeed service.
To begin we tell a little bit about the program interface. After the first start you will see the main window, which prompts you to select the data sources.
Pay attention to the description of each source, if the following is written next to it:
Source is designed to get history data … - then it means the ability to download historical data,
and if the following is written:
Source is designed to get market-data .. - then it means the ability to connect to the real-time data source and one's own history collection. Looking ahead, a similar process is often cheaper than a simple data purchase from a data vendor.
Now select the number of sources and try to get the market data. As such, we propose to use: Google (as a source of historical data), OECTrader (as a source of real-time data, which the broker gives you when opening an account), IQFeed (as a source of real-time data with the maximum number of concurrent subscriptions).
Click the check-boxes in the selected sources.
After you click OK, the program prompts you to enable additional features.
They are well documented in the program itself, so we will not go into a description of each of them.
In order to move forward it will be enough just to click OK without selecting anything in this window. If necessary, you can always restore this setting via the ADD -> Tools button and select the required.
After all the previous procedure, we have got added sources in the left pane, all of which must now be set up.
Google Finance
This is done simply by pressing the pen button,
that opens the settings window.
Quickly go over each of them to get a full understanding. So:
- Start date – the date from which Hydra will receive market data
- Time Offset – In this case, 1 means that the today data will not be downloaded. This is necessary in order not to download half a day, when the trading have not yet been completed.
- Weekend – weekends ignored when checkbox is selected.
- Time interval – Hydra downloads data in parts. This parameter allows to specify how large data part will be used. With a value of 30, the program will download data packages for 30 days.
- Header, work from, work until, work interval – settings for the program itself, the time interval during which the program should download data (from and to)
- Data directory – a directory in which the downloaded data will be stored, you can leave the default, or choose any other. We recommend to create a separate directory for each source, in order to not overwrite the data.
- Format - the format of the stored data. BIN supported – Hydra’s special format allows to get a unique compression ratio (2 bytes per tick, 7 bytes per order book) or well-known CSV (it is normal volume)
- Max.errors - the maximum number of errors in the source.
- Dependency - specifies the added task that must be completed before the current (in this case, it may be either IQFeed, or OEC Trader)
- Logging level - the level of logging.
Let's leave all the default data and choose the securities on which the data will be downloaded. Let's say we choose 5 tickers included in the S&P500 index, for example: MMM, AFL, GOOG, AAPl, T.
For this purpose, all these tools you need to add:
Since Google source does not support the automatic addition of the tools you should to add them to the database manually
Then repeat the procedure for each security and move them to the Selected section
Then simply connect the source
and press Start.
After the end of the downloading you have to get the following results
OpenECry (Gain Futures)
The market data source for the Gain Capital clients, which a broker gives you for free when opening an account. The source key settings in many ways similar to the settings that we have made in Google with the only difference that now you should enter your account username/password to access the data stream and select the correct address from which the data will come. Preset 3 possibilities: the test server (to which we connect and which is intended for developers), the simulator (server for demo accounts), the real server (the server that has the connection to the real market and real accounts). If you have an opened account, then your choice is the real server.
The CandleFromDate parameter is needed to specify the starting date from which you want to download the history in the form of candles.
Once it has been done, we add instruments by which we want to get data, because OEC supports automatic downloading and searching of instruments. We do it through the Download Securities button, and then adding them through the instrument code. Our selection ESZ6 (mini S&P) and NQZ6 (mini Nasdaq).
After adding you should get the same as shown in the picture.
Now turn your attention to the bottom part of the window. At the beginning of this article, we talked about the fact that Hydra supports downloading and storing a plurality of market data types, settings about what data to download can be done using the appropriate panel:
On Ticks - means that ticks will be downloaded
On Market Depth - means that order books will be downloaded
Candles has additional setting, by clicking the button you can select the candles types which will be downloaded, that's how it looks:
On Level 1 - means that the best bid/ask will be downloaded, as well as some fields of the ticker basic statistics.
Now just click the Start button, and Hydra will start to receive data and store them locally. In this case, both historical and real-time market data will be received.
Thus, you can collect and store the market data yourself, and then use them for testing your own strategies.
IQFeed
IQFeed provides as OpenECry, intraday data. But, in contrast to the OEC, IQFeed supports a very wide range of parallel subscriptions, as well as a much greater depth of history, both for candles and tick data.
The source settings similar to OpenEcry:
The CandleFromDate parameter is needed to specify the starting date from which you want to download the history in the form of candles.
The TicksFromDate parameter is needed to specify the starting date from which you want to download the history in the form of ticks.
All other steps are similar to those described above.
Data viewing
Now, after all the manipulations, you need to understand, and how to work with these data, and what the program can do else. Recall that data is stored in the directory where you downloaded them, but how to see what is a result.
For this we refer to the OEC Trader source from which a lot of data has been downloaded, select it and click the right mouse button.
Thereafter, we can choose in the menu the way to work with the instrument. Let's choose Market Depths and then in the window that opens, click the magnifying glass.
As a result, we will get a similar picture
Next, similarly we select Level 1, and then Candles 1 min, but in the window with candles we do not choose the magnifying glass, and Chart button. The results are shown in the figures below
That's the way you can view the data and work with them directly from the program.
We hope our product will help you in trading and development of profitable trading systems.