IEP aggregate operators

5 messages

IEP aggregate operators

Reply Threaded

Hi,

I'm trying to evaluate IEP for handling some processing. I'm having
trouble finding useful information or documentation.

For an example, I want to take a stream of price ticks and create a
stream of price bars. Conceptually, using a time based aggregator operator:
CREATE INPUT STREAM tick (symbol VARCHAR, price DOUBLE, shares INTEGER)
CREATE OUTPUT STREAM bar (symbol VARCHAR, bar_time TIMESTAMP, open
DOUBLE, high DOUBLE, low DOUBLE, close DOUBLE, volume INTEGER)
INSERT INTO bar SELECT symbol, aggregate_timestamp(), first(price),
max(price), min(price) last(price), sum(volume) FROM tick AGGREGATE 1min
ADVANCE 1min GROUP BY symbol

The most detailed information on the aggregators I have found is the
document in the Java CAPS documentation. [1] But it's not very detailed.

Are the available aggregate functions documented anywhere? Is it
possible to add new aggregate functions? How do I assign the time the
bar events are issued from the aggregator into one of the fields of the
event?

TIA
Malcolm

[1]
<http://docs.sun.com/app/docs/doc/820-3378/dsgn_iep-ops-aggr_t?l=en&a=view>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Prashant Bhagat

Re: IEP aggregate operators

Reply Threaded

Hi Malcolm,
You can find the Functional Specification on this page -
http://wiki.open-esb.java.net/Wiki.jsp?page=IEPSEDeveloper (the fist link)
You can also find documentation in Netbeans Help. Just open up the
dialog for defining the properties of an operator (double click) and
click on help.

HTH
Thanks

Malcolm Studd wrote:

> Hi,
>
> I'm trying to evaluate IEP for handling some processing. I'm having
> trouble finding useful information or documentation.
>
> For an example, I want to take a stream of price ticks and create a
> stream of price bars. Conceptually, using a time based aggregator
> operator:
> CREATE INPUT STREAM tick (symbol VARCHAR, price DOUBLE, shares INTEGER)
> CREATE OUTPUT STREAM bar (symbol VARCHAR, bar_time TIMESTAMP, open
> DOUBLE, high DOUBLE, low DOUBLE, close DOUBLE, volume INTEGER)
> INSERT INTO bar SELECT symbol, aggregate_timestamp(), first(price),
> max(price), min(price) last(price), sum(volume) FROM tick AGGREGATE
> 1min ADVANCE 1min GROUP BY symbol
>
> The most detailed information on the aggregators I have found is the
> document in the Java CAPS documentation. [1] But it's not very detailed.
>
> Are the available aggregate functions documented anywhere? Is it
> possible to add new aggregate functions? How do I assign the time the
> bar events are issued from the aggregator into one of the fields of
> the event?
>
> TIA
> Malcolm
>
> [1]
> <http://docs.sun.com/app/docs/doc/820-3378/dsgn_iep-ops-aggr_t?l=en&a=view>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

--
Prashant Bhagat
Open ESB Community (http://open-esb.org)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Prashant Bhagat

Re: IEP aggregate operators

Reply Threaded

Also, the StreamOutput operator has the facility to assign timestamp to
the outgoing events. You can use any function that is available in SQL
as well as any database specific functions that may be available. Of
course, that would tie you down to that database.

Prashant Bhagat wrote:

> Hi Malcolm,
> You can find the Functional Specification on this page -
> http://wiki.open-esb.java.net/Wiki.jsp?page=IEPSEDeveloper (the fist
> link)
> You can also find documentation in Netbeans Help. Just open up the
> dialog for defining the properties of an operator (double click) and
> click on help.
>
> HTH
> Thanks
>
> Malcolm Studd wrote:
>> Hi,
>>
>> I'm trying to evaluate IEP for handling some processing. I'm having
>> trouble finding useful information or documentation.
>>
>> For an example, I want to take a stream of price ticks and create a
>> stream of price bars. Conceptually, using a time based aggregator
>> operator:
>> CREATE INPUT STREAM tick (symbol VARCHAR, price DOUBLE, shares INTEGER)
>> CREATE OUTPUT STREAM bar (symbol VARCHAR, bar_time TIMESTAMP, open
>> DOUBLE, high DOUBLE, low DOUBLE, close DOUBLE, volume INTEGER)
>> INSERT INTO bar SELECT symbol, aggregate_timestamp(), first(price),
>> max(price), min(price) last(price), sum(volume) FROM tick AGGREGATE
>> 1min ADVANCE 1min GROUP BY symbol
>>
>> The most detailed information on the aggregators I have found is the
>> document in the Java CAPS documentation. [1] But it's not very
>> detailed.
>>
>> Are the available aggregate functions documented anywhere? Is it
>> possible to add new aggregate functions? How do I assign the time the
>> bar events are issued from the aggregator into one of the fields of
>> the event?
>>
>> TIA
>> Malcolm
>>
>> [1]
>> <http://docs.sun.com/app/docs/doc/820-3378/dsgn_iep-ops-aggr_t?l=en&a=view>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>

Malcolm Studd

Re: IEP aggregate operators

Reply Threaded

In reply to this post by Prashant Bhagat

Prashant Bhagat wrote:

You can find the Functional Specification on this page -
http://wiki.open-esb.java.net/Wiki.jsp?page=IEPSEDeveloper (the fist link)

I looked at that. It's a mathematical description of the underlying theory, not documentation on using the product.

You can also find documentation in Netbeans Help. Just open up the
dialog for defining the properties of an operator (double click) and
click on help.

This is the exact same text as I linked to. Which is not helpful if one already understands how to use the operator, but needs to know all the available functions.

What I'm looking for is a definitive list of all aggregate functions supported and documentation on using them.
Eg. "The supported aggregatefunctions are:"
* avg(field): takes an integer or double field and returns the arithmetic mean of all values. Returns a double.
* prior(field, count): takes any field type. Returns the value count events from the last event in the window. If there are fewer than count values in the window, returns null.
* ..."
And so on.

Malcolm

Prashant Bhagat

Re: IEP aggregate operators

Reply Threaded

Probably you missed my second email (copied below where I answered the
question on aggregate function.

"Also, the StreamOutput operator has the facility to assign timestamp to
the outgoing events. You can use any function that is available in SQL
as well as any database specific functions that may be available. Of
course, that would tie you down to that database."

Thanks

Malcolm Studd wrote:

> Prashant Bhagat wrote:
>
>> You can find the Functional Specification on this page -
>> http://wiki.open-esb.java.net/Wiki.jsp?page=IEPSEDeveloper (the fist link)
>>
>>
> I looked at that. It's a mathematical description of the underlying theory,
> not documentation on using the product.
>
>
>
>> You can also find documentation in Netbeans Help. Just open up the
>> dialog for defining the properties of an operator (double click) and
>> click on help.
>>
>>
> This is the exact same text as I linked to. Which is not helpful if one
> already understands how to use the operator, but needs to know all the
> available functions.
>
> What I'm looking for is a definitive list of all aggregate functions
> supported and documentation on using them.
> Eg. "The supported aggregatefunctions are:"
> * avg(field): takes an integer or double field and returns the arithmetic
> mean of all values. Returns a double.
> * prior(field, count): takes any field type. Returns the value count
> events from the last event in the window. If there are fewer than count
> values in the window, returns null.
> * ..."
> And so on.
>
> Malcolm
>