Questions/Concerns when using Marketcetera as core platform

[rivercode]

Hi,
 
First of all...congratulations on creating and launching an institutional level open-source order routing and execution system. While it is still a little rough a round the edges, which is to be expected with an alpha/beta version it really has real potential to shake up the industry.
 
I am reviewing platforms (both commercial and open source) for fully automated/algorithmic trading to use as our core technology upon which we will grow. There are many attractions to an open-source solution as opposed to a very expensive and closed source solution.  In the open source area, yours does seem to be the best in terms of future potential though ActiveQuant is more mature.
 
The following is a list of questions/issues that I have todate, which will help me make a decision. Instead of emailing them to you privately, I have posted them here as I am sure others will have the same concerns/issues.
 
Please answer where you can ( I appreciate there are limits due to confidentiality issues).
 
1. DataFeeds:
 
   Besides Activ do you have any other data feeds integrated or will be in the near future ?
 
   How was Activ added into the platform ? Specifically, how does the market data enter platform and how is it handled internally...eg. through the Quickfix engine or directly to the JMS (ActiveMQ) ?  Just trying to understand how you are architecting it.
 
  Based on your experience, what is your approx estimate for a competent developer to add and test a new datafeed to the platform, assuming a clearn API from the vendor ?
 
2. Production Use:
 
  Can you tell us about any production installations of MarketCetera ? Can you give a summary of what their focus and type of trading is, eg. Asset Class (eg. Equities) and type of trading (High Frequency, Intraday, etc)
 
3. CEP Integration

   The grapevine mentions that Esper (esper.codehaus.org) will be integrated with Marketetcera...can you comment on that ?
 
4. Release 0.6:

   When do you expect the next release out ?
 
5. Support and Business Revenue:

   The support forums look kind of dead for the last month (no entry since Sept 28), which is always a concern when evaluating open source software which will be critical to the business.  Can you comment on the health of Marketcetera as a business and your capital situation...while I realize this is a delicate question in an open forum, it is also a critical question for a business that wants to rely upon Marketcetera.
 
6. Scalablity:

    Do you have any information you can share in terms of scalability, actual usage, message throughput ? While this obviously depends upon the underlying hardware server architecture, maybe you can give an example of an implementation already done ?

Thank you.

[toli]

Hi,

Thanks for the interest in Marketcetera and your kind words. It's nice
to see the validation of our beliefs that leveraging open source is
the best approach to developing institutional trading systems.

Let me address your questions:

> to a very expensive and closed source solution.  In the open source area,
> yours does seem to be the best in terms of future potential though
> ActiveQuant is more mature.

It's always great to have competition, even in open source - it leads
to better products for everybody. With respect to ActiveQuant, it
depends on what kind of functionality you want. They certainly have
more backtesting, but I believe that Marketcetera is better positioned
to work "out of the box", with FIX connections to your brokers and an
easier way to stage your strategies (for non-developers).

> The following is a list of questions/issues that I have todate, which will
> help me make a decision. Instead of emailing them to you privately, I have
> posted them here as I am sure others will have the same concerns/issues.
> Please answer where you can ( I appreciate there are limits due to
> confidentiality issues).

no problem, we are open source - part of the way we are different from
your usual proprietary vendors is that we firmly believe in openness
and transparency, and in giving you full control over your trading
platform.

> 1. DataFeeds:
>   Besides Activ do you have any other data feeds integrated or will be in
> the near future ?

Right now it's just Activ and OpenTick (arguably, it's not
institutional-grade). We've done some others on professional services
basis (such as FIX-based feeds for currencies). All the future data
feeds are customer-driven, so we'll be adding new ones in the future.

>   How was Activ added into the platform ? Specifically, how does the market
> data enter platform and how is it handled internally...eg. through the
> Quickfix engine or directly to the JMS (ActiveMQ) ?  Just trying to
> understand how you are architecting it.

The data feeds bypass JMS completely to avoid taking a performance
hit. Both Activ and OpenTick are a direct connection to their servers
(or local workstation if you have that setup) and the market data is
fed directly into strategies.

>  Based on your experience, what is your approx estimate for a competent
> developer to add and test a new datafeed to the platform, assuming a clearn
> API from the vendor ?

Most of the complexity here is usually figuring out the vendor's APIs,
unfortunately. Parsing the incoming data isn't too difficult, figuring
out how to register for the right set of data (symbols, options
chains, etc) is. Another surprisingly non-trivial amount of time is
spent on red tape and administrative overhead.
Another slowdown is availability of the test data - if the only way to
test your adapter is during market hours, then it severely cuts down
on your ability to write unit tests and test things.

All in all, conservatively it should probably take about 2 weeks to
add another data feed in.

> 2. Production Use:
>
>  Can you tell us about any production installations of MarketCetera ? Can
> you give a summary of what their focus and type of trading is, eg. Asset
> Class (eg. Equities) and type of trading (High Frequency, Intraday, etc)

We have a few implementations that we have setup ourselves that are
equity, equity options and currencies. I'm not sure what the frequency
of their trades are, but I believe it's fairly modest - possibly in
the 100s of trades/day.

There are also quite a few community implementations "in the wild"
that we know of, and they cover the same equity/options/currencies
classes, and I believe there may even be some people trading futures
although the platform as-is doesn't have good support for it yet.
I can't be more specific here unfortunately.

> 3. CEP Integration
>   The grapevine mentions that Esper (esper.codehaus.org) will be integrated
> with Marketetcera...can you comment on that ?

That is correct - Esper is going to be integrated in 1.0 (early
December). We'll expose the EPL query language, and you should be able
to use CEP in your strategies, both in Photon and in "headless mode"
running on top of our Strategy Agent (in development right now).

> 4. Release 0.6:
>   When do you expect the next release out ?

0.6.0 was out mid-August - it was an internal release to a few
selected beta users of the Activ market data feed.
We plan on having a release of 0.9.0 later this week (or early next
week), dependent on when we finish migration of all the documentation
to Jira/Confluence.
We'll be sending out an email to the lists when 0.9.0 is out - but you
can also find all the tags for prior releases at
http://trac.marketcetera.org/trac.fcgi/browser/tags

The 0.9.0 has support for notifications in Photon (fading popups that
you can call from Strategies to give visual notifications), migration
to Eclipse 3.4.1 (to support notifications), and a lot of
infrastructure work for the upcoming Strategy Agent support.

The community version of the 1.0 is slated for early December, with
full Strategy Agent support (headless strategies), CEP integration,
Photon restart/recovery, and a few other features.

> 5. Support and Business Revenue:
>   The support forums look kind of dead for the last month (no entry since
> Sept 28), which is always a concern when evaluating open source software
> which will be critical to the business.  Can you comment on the health of
> Marketcetera as a business and your capital situation...while I realize this
> is a delicate question in an open forum, it is also a critical question for
> a business that wants to rely upon Marketcetera.

I don't want to be overly glib, but I think quite a few things outside
of our control have been happening in finance in the past month, and
I'm not surprised that there's been a slowdown on our message board
while everybody sorts out what is going on in their professional
lives.

Marketcetera as a business is still strong - open source companies
have historically benefited and came out strong in times of economic
downturn (the dot-com bubble, for example) - see the recent statements
by Red Hat CEO Jim Whitehurst
(http://news.zdnet.com/2424-9595_22-241870.html). While the IT budgets
may shrink, the demands from IT desks at trading firms will remain the
same if not grow. Marketcetera can offer very high value proposition
compared to proprietary vendors, especially since we not only offer
full control and flexibility, but also zero software startup costs.

As far as support, we'll be offering different support packages with
various SLAs coinciding with the release of the 1.0 version.  I can
put you in touch with our sales guy if you want to know more details.

> 6. Scalablity:
>    Do you have any information you can share in terms of scalability,
> actual usage, message throughput ? While this obviously depends upon the
> underlying hardware server architecture, maybe you can give an example of an
> implementation already done ?

We'll be publishing better numbers for throughput with the 1.0
release. The anecdotal evidence right now is that ActiveMQ supports on
the order of 10,000 messages/sec, and QuickFIX/J is somewhere between
5,000-6,000 msgs/sec.
Certainly, the first step to speed up Marketcetera would be to
throttle down the log level in in the ORS - it is currently setup to
log a lot.

Let me know if you have any other questions.


--
Toli Kuznets
http://www.marketcetera.com: Open-Source Trading Platform
download.run.trade.
_______________________________________________
Marketcetera Developers mailing list
Marketcetera Documentation: http://trac.marketcetera.org
To unsubscribe, go to
http://lists.marketcetera.org/cgi-bin/mailman/listinfo/m-etc-dev or send an email to [hidden email].

[rivercode]

Hi,

Thank you for replying to my previous email.

A couple more Qs based on your answers and some others.


>The data feeds bypass JMS completely to avoid taking a performance
>hit. Both Activ and OpenTick are a direct connection to their servers
>(or local workstation if you have that setup) and the market data is
>fed directly into strategies.

I may have missed something earlier when looking at the platform...if I want to prototype an idea in Photon, how do I get the market data to it ? There has to be a connection from MarketCetera to the datafeed API. Currently Photon gets market data from your demo exchange, if I am adding a new 3rd party API to MarketCetera, where would I build this so that Photon and the Strategy Agent (ver 1.0 release) get the market data ? I have the feeling that I may be asking a dumb question and have overlooked the obvious....


>That is correct - Esper is going to be integrated in 1.0 (early
>December). We'll expose the EPL query language, and you should be able
>to use CEP in your strategies, both in Photon and in "headless mode"
>running on top of our Strategy Agent (in development right now).

Will I be able to use Esper's EPL query language directly from Photon and Strategy Agent ? For example (not in EPL...just generic):
        Open Stream x
        Open Stream y

        Do stuff on x and y

        If "Pattern Match" then
                Send Order // Order will be sent via MarketCetera


If this is the case, will Photon be able to display my order positions/fills/cancels/etc sent from Photon/Strategy agent with signals generated by Esper ?

Quite honestly, if you get this working and it easy to setup/implement you are going to give a lot of vendors major heartache and cause an explosion of use in the CEP/ESP field.

>The community version of the 1.0 is slated for early December, with
>full Strategy Agent support (headless strategies), CEP integration,
>Photon restart/recovery, and a few other features.

You say "Community version"...does this mean release to selected members of the community or for general testing/use by the public ?

Given that early December is only about 35-40 days away...are you confident of this timeline ?

> Strategy Agent in version 1.0

Can you tell us more about the Strategy Agent (or is there a link ?)...for example what language would I use to develop strategies in ?


Thank you for your time.

[Colin DuPlantis-2]

rivercode wrote:

> Hi,
>
> Thank you for replying to my previous email.
>
> A couple more Qs based on your answers and some others.
>
>
>> The data feeds bypass JMS completely to avoid taking a performance
>> hit. Both Activ and OpenTick are a direct connection to their servers
>> (or local workstation if you have that setup) and the market data is
>> fed directly into strategies.
>
> I may have missed something earlier when looking at the platform...if I want
> to prototype an idea in Photon, how do I get the market data to it ? There
> has to be a connection from MarketCetera to the datafeed API. Currently
> Photon gets market data from your demo exchange, if I am adding a new 3rd
> party API to MarketCetera, where would I build this so that Photon and the
> Strategy Agent (ver 1.0 release) get the market data ? I have the feeling
> that I may be asking a dumb question and have overlooked the obvious....

If you want to add a market data stream to Photon, for releases prior to
1.0, the basic steps are as follows:

1) Implement subclasses of:

 a) AbstractMarketDataFeed - this is the main class for a market data
provider.  If you implement the abstract methods required by the
superclass, that should be sufficient to get the flow going.

 b) AbstractMarketDataFeedFactory - provides an instance of (a)

 c) AbstractMarketDataFeedCredentials - encapsulates the data necessary
to initiate the flow, username, password, source URL, that kind of
thing.  The details vary depending on the nature of your market data
provider

 d) AbstractMarketDataFeedToken - copy an existing implementation
renamed for your feed

 e) AbstractMessageTranslator - translates a FIX market data request
(msgtype V) to whatever format your feed needs for data requests

 f) AbstractEventTranslator - translates responses from your feed to
subclasses of org.marketcetera.event.EventBase.  For now, Photon
requires the FIX member variable of EventBase objects to be present.
This goes away in 1.0.

 g) Create a Photon plug-in that can call instantiate and call your
feed.  Have a look at the org.marketcetera.marketdata Photon plug-in for
an example.  This is the plug-in that wraps the Marketcetera feed.
Notice in particular the two extensions this plug-in implements:
org.marketcetera.photon.marketdatafeed and
org.eclipse.ui.preferencePages.  The latter is optional, but is a good
example for how to create a preference page if necessary.

 h) Add your plug-in to plugin_customization.ini in the list of plug-ins
under the category org.marketcetera.photon/marketdata.startup

This is a lot to digest, I know.  Let me know if you have any questions.
 This process changes somewhat in 1.0.  Many of the basics are the same,
but the structure will be a little different.

>> That is correct - Esper is going to be integrated in 1.0 (early
>> December). We'll expose the EPL query language, and you should be able
>> to use CEP in your strategies, both in Photon and in "headless mode"
>> running on top of our Strategy Agent (in development right now).
>
> Will I be able to use Esper's EPL query language directly from Photon and
> Strategy Agent ? For example (not in EPL...just generic):
> Open Stream x
> Open Stream y
>
> Do stuff on x and y
>
> If "Pattern Match" then
> Send Order // Order will be sent via MarketCetera
>
>
> If this is the case, will Photon be able to display my order
> positions/fills/cancels/etc sent from Photon/Strategy agent with signals
> generated by Esper ?

We're still working out the details for this in some respect.  What is
likely to be the case is that you will be able to specify an Esper query
in EPL from a strategy.

<snip>

>> The community version of the 1.0 is slated for early December, with
>> full Strategy Agent support (headless strategies), CEP integration,
>> Photon restart/recovery, and a few other features.
>
> You say "Community version"...does this mean release to selected members of
> the community or for general testing/use by the public ?

I believe that the plan is that our community version will be available
for generate download.

> Given that early December is only about 35-40 days away...are you confident
> of this timeline ?

As with any plan, it is always somewhat subject to change, but, with the
information we have today, we will meet this deadline.

>> Strategy Agent in version 1.0
>
> Can you tell us more about the Strategy Agent (or is there a link ?)...for
> example what language would I use to develop strategies in ?

The documentation and designs aren't available to the public yet, but
there will be something available to the public at the time of release
or very soon thereafter.

In general, though, Strategy Agent refers to the Marketcetera product
that runs in headless (non-Photon) mode.  We will support a
configuration that runs without a GUI.  The components of the Strategy
Agent are market data, CEP (Complex Event Processor, either with Esper
or without), strategy execution, and the ORS.  For the community release
of 1.0 you will be able to write strategies in Java and Ruby.  This is
all according to the current plan which is, as always, subject to change.
_______________________________________________
Marketcetera Developers mailing list
Marketcetera Documentation: http://trac.marketcetera.org
To unsubscribe, go to
http://lists.marketcetera.org/cgi-bin/mailman/listinfo/m-etc-dev or send an email to [hidden email].