On Tue, 12 Feb 2008, David Brown wrote:
> Could I volunteer to work on the historical database
> management tool? Sun finished buying MySQL about three weeks
> ago, so I'm guessing we'd want to stick to that for an
> opensource approach.
MySQL has long been Free and Open Source Software before Sun.
We use MySQL already for the trading shim for the History Bar
store, down to per second granularity:
mysql> describe HistoryBar ;
+----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+------------------+------+-----+---------+----------------+
| uid | int(10) unsigned | NO | PRI | NULL | auto_increment |
| cid | int(10) unsigned | NO | MUL | NULL | |
| bid | int(10) unsigned | NO | MUL | NULL | |
| time | datetime | NO | | NULL | |
| open | decimal(10,4) | NO | | NULL | |
| high | decimal(10,4) | NO | | NULL | |
| low | decimal(10,4) | NO | | NULL | |
| close | decimal(10,4) | NO | | NULL | |
| vol | int(11) | NO | | NULL | |
| wap | decimal(10,4) | NO | | NULL | |
| has_gaps | tinyint(1) | NO | | NULL | |
+----------+------------------+------+-----+---------+----------------+
11 rows in set (0.00 sec)
uid is a Unique ID
cid is the underlying Contract ID
bid is the BarWidth ID
Other approaches exist; beancounter previously used OHLC daily
store in postgresql or mysgl, and recently cut over to sqlite;
and so forth. It is possible to add a field, to make it
extensible to sub-second granualrity (IB provides raw MD data
faster), but it is not at all clear that all that is our there
is marshalled together even within the one second timeframe we
use here.
Raw tickstreams and book delta data are dense (frequent)
enough that they may not well be suited to the
insert/retrieval load of a formal SQLish backend, from our
analysis
> Also, is it possible to also integrate IQ-Feed and / or
> ESignal as alternative datafeeds. Activ is rather
> expensive, right?
Of course with computers, the answer is always 'yes, but ...
how deep is your wallet?'
> A couple more questions, perhaps better sent in a separate
> mail. What would it take to integrate futures? What would
> it take to integrate options into the core database?
We presently handle all IB datatypes -- STK, IND, OPT, FUT,
CASH, FOP through the 'cid' normalization field
> There are a few nuances when it comes to either of those,
> such as handling corporate events. I'd also like to know if
> it might be possible to integrate a general "product"
> (replace the word produce with equity, option, whatever)
> approach as a higher abstraction. This would allow more
> complicated baskets to be handled seamlessly in the future.
> For instance, a core strategy I use is highly focused on a
> basket, typically long one Single Stock Future versus short
> a different Single Stock Future. Sometimes I use multiple
> options in place of futures. This type of abstraction would
> need to be added as a base setup.
You may wish to look at the trading shim's documentation and
schema.
> I have plenty of more ideas, but this is alot for one email
> and one instant. Looking forward to hearing your thoughts.
-- Russ herrold
_______________________________________________
m-etc-users mailing list
[hidden email]
http://lists.marketcetera.org/cgi-bin/mailman/listinfo/m-etc-users
