Nabble - Netty User Group

Re: How to implement Streaming server like The Bayeux Protocol

2009-11-24T09:02:32Z

Sorry i was not been clear.

That is what i want to do.

I am using xmlhttprequst.multipart to do the streaming, so once a client connects to get the data
whole day the connection is kept live ( the chunkwriter never returns).

An this is working fine for two users but as soon as there is a third user that connects he gets blocked.
Since i have set cached thread pool for boss and worker threads should i think it should work fine for n no of users until we run out of os threads or EXecutor cannot create any more thread???
NioServerSocketChannelFactory(Executors.newCachedThreadPool(),Executors.newCachedThreadPool()));

I also know for this mode of operation netty will fallback to one to one thread per client-server connection but i need netty to do it as i am using netty for other purpose already and i cannot mix any other libary ( jetty/mina).

Trustin Lee wrote:

Your ChunkedInput implementation never terminates since hasNextChunk()
always return true. Refer to the source code of the existing
ChunkedInput implementations to figure out how to implement it
correctly.

— Trustin Lee, http://gleamynode.net/

On Tue, Nov 24, 2009 at 11:51 PM, sumitatgharkikhoj
<mailtosumitsingh@gmail.com> wrote:
>
> chunked writer is same as above mentioned.
>
>
> sumitatgharkikhoj wrote:
>>
>> Hi Trustin, i replaced the line in the static file server example:
>> //ChannelFuture writeFuture = ch.write(new ChunkedFile(raf, 0,
>> fileLength, 8192));
>> with line :
>> ChannelFuture writeFuture = e.getChannel().write(new
>> ChunkedInput() {
>>
>> my server still getting blocked after two requests all other requests get
>> piled up.
>> What i did:
>> 1) Took the example httpstatic file server
>> 2) Instead of using chunked file use my own chunked input
>> 3) Opened three different browsers: firefox, chrmoe,ie ( to make sure the
>> test does not gets affected by browser)
>> 4) First two requests succeed and data starts streaming to both firefox
>> and chrome but ie request does not event reach my handler.
>> 5) When i escape or close one of the browser the third request goes
>> through and my static file server handler gets the response.
>>
>> I still get the same results the server gets blocked.
>>
>>
>>
>>
>> Trustin Lee wrote:
>>>
>>> On Tue, Nov 24, 2009 at 1:02 PM, sumitatgharkikhoj
>>> <mailtosumitsingh@gmail.com> wrote:
>>>>
>>>> First of all thank you for the nice framework, forum and documentation.
>>>> My problem is i am trying to write a http server with small footprint (
>>>> will
>>>> go as a component in bigger framework) but it needs to have streaming
>>>> support .
>>>> Different clients will connect at different times ranging from 100-1000
>>>> clients max and this server will be used for both streaming and short
>>>> data.
>>>> the data is streamed using XHR multipart request and continously (24/7)
>>>> .
>>>> But when i used the example code (please see below) the server blocks
>>>> after
>>>> first thread.
>>>
>>> In your handler, you check if the returned future.isSuccess() is true
>>> or false right after calling write(). It will be almost always false
>>> because the write operation is asynchronous. You are using
>>> ChunkedInput, so I'd recommend you take a look into the
>>> HttpStaticFileServerHandler.
>>>
>>>> Other option that i tried was jetty with continuation or cometd but i am
>>>> more inclined about netty (since i want to replace my mina udp layer
>>>> with
>>>> mina and maintain one single library ).
>>>>
>>>> Any help will be appreciated, I have my machine available via webex to
>>>> debug
>>>> as well.
>>>>
>>>>
>>>> my server gets block at second request in the following code :
>>>>
>>>> 1) in my http server:
>>>> ServerBootstrap bootstrap = new ServerBootstrap(new
>>>> NioServerSocketChannelFactory(Executors.newCachedThreadPool(),Executors.newCachedThreadPool()));
>>>> bootstrap.setPipelineFactory(new
>>>> HttpServerPipelineFactory(requestHandler));
>>>> bootstrap.bind(new InetSocketAddress(8080));
>>>> 2) in my pipelinefactory:
>>>> public ChannelPipeline getPipeline() throws Exception {
>>>> ChannelPipeline pipeline = pipeline();
>>>> pipeline.addLast("decoder", new HttpRequestDecoder());
>>>> pipeline.addLast("encoder", new HttpResponseEncoder());
>>>> pipeline.addLast("chunkedWriter", new ChunkedWriteHandler());
>>>> pipeline.addLast("handler", requestHandler);
>>>> return pipeline;
>>>> 3)my http request handler sends event using following code
>>>> private void writeResponse(MessageEvent e) {
>>>> ChannelBuffer buf =
>>>> ChannelBuffers.copiedBuffer(responseContent.toString(), "UTF-8");
>>>> buf.setBytes(0, responseContent.toString().getBytes() );
>>>> responseContent.setLength(0);
>>>> boolean close =
>>>> HttpHeaders.Values.CLOSE.equalsIgnoreCase(request
>>>> .getHeader(HttpHeaders.Names.CONNECTION))
>>>> ||
>>>> request.getProtocolVersion().equals(HttpVersion.HTTP_1_0)
>>>> &&
>>>> !HttpHeaders.Values.KEEP_ALIVE.equalsIgnoreCase(request
>>>>
>>>> .getHeader(HttpHeaders.Names.CONNECTION));
>>>>
>>>> HttpResponse response = new
>>>> DefaultHttpResponse(HttpVersion.HTTP_1_1,
>>>> HttpResponseStatus.OK);
>>>> //response.setContent(buf);
>>>> //response.setHeader(HttpHeaders.Names.CONTENT_TYPE,
>>>> "text/html;
>>>> charset=UTF-8");
>>>> response.setHeader(HttpHeaders.Names.CONTENT_TYPE,
>>>> "multipart/x-mixed-replace;boundary=XXoXoX");
>>>> //if (!close)
>>>> {
>>>> response.setHeader("Connection", "Keep-Alive");
>>>> response.setHeader("Cache-Control", "no-cache");
>>>> response.setHeader("Access-Control-Allow-Origin",
>>>> "*");
>>>> }
>>>>
>>>> String cookieString =
>>>> request.getHeader(HttpHeaders.Names.COOKIE);
>>>> if (cookieString != null) {
>>>> CookieDecoder cookieDecoder = new
>>>> CookieDecoder();
>>>> Set<Cookie> cookies =
>>>> cookieDecoder.decode(cookieString);
>>>> if (!cookies.isEmpty()) {
>>>> CookieEncoder cookieEncoder = new
>>>> CookieEncoder(true);
>>>> for (Cookie cookie : cookies) {
>>>> cookieEncoder.addCookie(cookie);
>>>> }
>>>>
>>>> response.addHeader(HttpHeaders.Names.SET_COOKIE,
>>>> cookieEncoder.encode());
>>>> }
>>>> }
>>>> ChannelFuture future =null;
>>>> future = e.getChannel().write(response);
>>>> String temp ="";
>>>>
>>>> if(future.isSuccess()==false)
>>>> return;
>>>>
>>>> e.getChannel().write(new ChunkedInput() {
>>>>
>>>> int i = 0;
>>>> public Object nextChunk() throws Exception {
>>>> i++;
>>>> String temp ="";
>>>> //temp ="--XXoXoX\r\n";
>>>> temp = "Content-Type: text/plain
>>>> \r\n\r\n";
>>>> temp += "{\"data\":\""+i+"\"}";
>>>> temp +="\r\n--XXoXoX\r\n";
>>>> ChannelBuffer buf2 =
>>>> ChannelBuffers.copiedBuffer(temp.toString(),
>>>> "UTF-8");
>>>> Thread.sleep(100);
>>>> return buf2;
>>>> }
>>>>
>>>> @Override
>>>> public boolean hasNextChunk() throws Exception {
>>>> return true;
>>>> }
>>>>
>>>> @Override
>>>> public void close() throws Exception {
>>>> // TODO Auto-generated method stub
>>>>
>>>> }
>>>> });
>>>>
>>>> if(close){
>>>> future.addListener(ChannelFutureListener.CLOSE);
>>>> }
>>>> }
>>>>
>>>> --
>>>> View this message in context:
>>>> http://n2.nabble.com/How-to-implement-Streaming-server-like-The-Bayeux-Protocol-tp4055722p4055722.html
>>>> Sent from the Netty User Group mailing list archive at Nabble.com.
>>>> _______________________________________________
>>>> netty-users mailing list
>>>> netty-users@lists.jboss.org
>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>
>>>
>>> — Trustin Lee, http://gleamynode.net/
>>>
>>> _______________________________________________
>>> netty-users mailing list
>>> netty-users@lists.jboss.org
>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>
>>>
>>
>>
>
> --
> View this message in context: http://n2.nabble.com/How-to-implement-Streaming-server-like-The-Bayeux-Protocol-tp4055722p4058231.html
> Sent from the Netty User Group mailing list archive at Nabble.com.
>
> _______________________________________________
> netty-users mailing list
> netty-users@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users
>

_______________________________________________
netty-users mailing list
netty-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/netty-users

Re: How to implement Streaming server like The Bayeux Protocol

2009-11-24T08:21:13Z

Your ChunkedInput implementation never terminates since hasNextChunk()
always return true. Refer to the source code of the existing
ChunkedInput implementations to figure out how to implement it
correctly.

— Trustin Lee, http://gleamynode.net/

On Tue, Nov 24, 2009 at 11:51 PM, sumitatgharkikhoj
<[hidden email]> wrote:

>
> chunked writer is same as above mentioned.
>
>
> sumitatgharkikhoj wrote:
>>
>> Hi Trustin, i replaced the line in the static file server example:
>> //ChannelFuture writeFuture = ch.write(new ChunkedFile(raf, 0,
>> fileLength, 8192));
>> with line :
>> ChannelFuture writeFuture = e.getChannel().write(new
>> ChunkedInput() {
>>
>> my server still getting blocked after two requests all other requests get
>> piled up.
>> What i did:
>> 1) Took the example httpstatic file server
>> 2) Instead of using chunked file use my own chunked input
>> 3) Opened three different browsers: firefox, chrmoe,ie ( to make sure the
>> test does not gets affected by browser)
>> 4) First two requests succeed and data starts streaming to both firefox
>> and chrome but ie request does not event reach my handler.
>> 5) When i escape or close one of the browser the third request goes
>> through and my static file server handler gets the response.
>>
>> I still get the same results the server gets blocked.
>>
>>
>>
>>
>> Trustin Lee wrote:
>>>
>>> On Tue, Nov 24, 2009 at 1:02 PM, sumitatgharkikhoj
>>> <[hidden email]> wrote:
>>>>
>>>> First of all thank you for the nice framework, forum and documentation.
>>>> My problem is i am trying to write a http server with small footprint (
>>>> will
>>>> go as a component in bigger framework) but it needs to have streaming
>>>> support .
>>>> Different clients will connect at different times ranging from 100-1000
>>>> clients max and this server will be used for both streaming and short
>>>> data.
>>>> the data is streamed using XHR multipart request and continously (24/7)
>>>> .
>>>> But when i used the example code (please see below) the server blocks
>>>> after
>>>> first thread.
>>>
>>> In your handler, you check if the returned future.isSuccess() is true
>>> or false right after calling write(). It will be almost always false
>>> because the write operation is asynchronous. You are using
>>> ChunkedInput, so I'd recommend you take a look into the
>>> HttpStaticFileServerHandler.
>>>
>>>> Other option that i tried was jetty with continuation or cometd but i am
>>>> more inclined about netty (since i want to replace my mina udp layer
>>>> with
>>>> mina and maintain one single library ).
>>>>
>>>> Any help will be appreciated, I have my machine available via webex to
>>>> debug
>>>> as well.
>>>>
>>>>
>>>> my server gets block at second request in the following code :
>>>>
>>>> 1) in my http server:
>>>> ServerBootstrap bootstrap = new ServerBootstrap(new
>>>> NioServerSocketChannelFactory(Executors.newCachedThreadPool(),Executors.newCachedThreadPool()));
>>>> bootstrap.setPipelineFactory(new
>>>> HttpServerPipelineFactory(requestHandler));
>>>> bootstrap.bind(new InetSocketAddress(8080));
>>>> 2) in my pipelinefactory:
>>>> public ChannelPipeline getPipeline() throws Exception {
>>>> ChannelPipeline pipeline = pipeline();
>>>> pipeline.addLast("decoder", new HttpRequestDecoder());
>>>> pipeline.addLast("encoder", new HttpResponseEncoder());
>>>> pipeline.addLast("chunkedWriter", new ChunkedWriteHandler());
>>>> pipeline.addLast("handler", requestHandler);
>>>> return pipeline;
>>>> 3)my http request handler sends event using following code
>>>> private void writeResponse(MessageEvent e) {
>>>> ChannelBuffer buf =
>>>> ChannelBuffers.copiedBuffer(responseContent.toString(), "UTF-8");
>>>> buf.setBytes(0, responseContent.toString().getBytes() );
>>>> responseContent.setLength(0);
>>>> boolean close =
>>>> HttpHeaders.Values.CLOSE.equalsIgnoreCase(request
>>>> .getHeader(HttpHeaders.Names.CONNECTION))
>>>> ||
>>>> request.getProtocolVersion().equals(HttpVersion.HTTP_1_0)
>>>> &&
>>>> !HttpHeaders.Values.KEEP_ALIVE.equalsIgnoreCase(request
>>>>
>>>> .getHeader(HttpHeaders.Names.CONNECTION));
>>>>
>>>> HttpResponse response = new
>>>> DefaultHttpResponse(HttpVersion.HTTP_1_1,
>>>> HttpResponseStatus.OK);
>>>> //response.setContent(buf);
>>>> //response.setHeader(HttpHeaders.Names.CONTENT_TYPE,
>>>> "text/html;
>>>> charset=UTF-8");
>>>> response.setHeader(HttpHeaders.Names.CONTENT_TYPE,
>>>> "multipart/x-mixed-replace;boundary=XXoXoX");
>>>> //if (!close)
>>>> {
>>>> response.setHeader("Connection", "Keep-Alive");
>>>> response.setHeader("Cache-Control", "no-cache");
>>>> response.setHeader("Access-Control-Allow-Origin",
>>>> "*");
>>>> }
>>>>
>>>> String cookieString =
>>>> request.getHeader(HttpHeaders.Names.COOKIE);
>>>> if (cookieString != null) {
>>>> CookieDecoder cookieDecoder = new
>>>> CookieDecoder();
>>>> Set<Cookie> cookies =
>>>> cookieDecoder.decode(cookieString);
>>>> if (!cookies.isEmpty()) {
>>>> CookieEncoder cookieEncoder = new
>>>> CookieEncoder(true);
>>>> for (Cookie cookie : cookies) {
>>>> cookieEncoder.addCookie(cookie);
>>>> }
>>>>
>>>> response.addHeader(HttpHeaders.Names.SET_COOKIE,
>>>> cookieEncoder.encode());
>>>> }
>>>> }
>>>> ChannelFuture future =null;
>>>> future = e.getChannel().write(response);
>>>> String temp ="";
>>>>
>>>> if(future.isSuccess()==false)
>>>> return;
>>>>
>>>> e.getChannel().write(new ChunkedInput() {
>>>>
>>>> int i = 0;
>>>> public Object nextChunk() throws Exception {
>>>> i++;
>>>> String temp ="";
>>>> //temp ="--XXoXoX\r\n";
>>>> temp = "Content-Type: text/plain
>>>> \r\n\r\n";
>>>> temp += "{\"data\":\""+i+"\"}";
>>>> temp +="\r\n--XXoXoX\r\n";
>>>> ChannelBuffer buf2 =
>>>> ChannelBuffers.copiedBuffer(temp.toString(),
>>>> "UTF-8");
>>>> Thread.sleep(100);
>>>> return buf2;
>>>> }
>>>>
>>>> @Override
>>>> public boolean hasNextChunk() throws Exception {
>>>> return true;
>>>> }
>>>>
>>>> @Override
>>>> public void close() throws Exception {
>>>> // TODO Auto-generated method stub
>>>>
>>>> }
>>>> });
>>>>
>>>> if(close){
>>>> future.addListener(ChannelFutureListener.CLOSE);
>>>> }
>>>> }
>>>>
>>>> --
>>>> View this message in context:
>>>> http://n2.nabble.com/How-to-implement-Streaming-server-like-The-Bayeux-Protocol-tp4055722p4055722.html
>>>> Sent from the Netty User Group mailing list archive at Nabble.com.
>>>> _______________________________________________
>>>> netty-users mailing list
>>>> [hidden email]
>>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>>
>>>
>>> — Trustin Lee, http://gleamynode.net/
>>>
>>> _______________________________________________
>>> netty-users mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>>
>>>
>>
>>
>
> --
> View this message in context: http://n2.nabble.com/How-to-implement-Streaming-server-like-The-Bayeux-Protocol-tp4055722p4058231.html
> Sent from the Netty User Group mailing list archive at Nabble.com.
>
> _______________________________________________
> netty-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/netty-users
>

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

— Trustin Lee, http://gleamynode.net/

Re: How to implement Streaming server like The Bayeux Protocol

2009-11-24T06:51:09Z

chunked writer is same as above mentioned.

sumitatgharkikhoj wrote:

Hi Trustin, i replaced the line in the static file server example:
//ChannelFuture writeFuture = ch.write(new ChunkedFile(raf, 0, fileLength, 8192));
with line :
ChannelFuture writeFuture = e.getChannel().write(new ChunkedInput() {

my server still getting blocked after two requests all other requests get piled up.
What i did:
1) Took the example httpstatic file server
2) Instead of using chunked file use my own chunked input
3) Opened three different browsers: firefox, chrmoe,ie ( to make sure the test does not gets affected by browser)
4) First two requests succeed and data starts streaming to both firefox and chrome but ie request does not event reach my handler.
5) When i escape or close one of the browser the third request goes through and my static file server handler gets the response.

I still get the same results the server gets blocked.

Trustin Lee wrote:

On Tue, Nov 24, 2009 at 1:02 PM, sumitatgharkikhoj
<mailtosumitsingh@gmail.com> wrote:
>
> First of all thank you for the nice framework, forum and documentation.
> My problem is i am trying to write a http server with small footprint ( will
> go as a component in bigger framework) but it needs to have streaming
> support .
> Different clients will connect at different times ranging from 100-1000
> clients max and this server will be used for both streaming and short data.
> the data is streamed using XHR multipart request and continously (24/7) .
> But when i used the example code (please see below) the server blocks after
> first thread.

In your handler, you check if the returned future.isSuccess() is true
or false right after calling write(). It will be almost always false
because the write operation is asynchronous. You are using
ChunkedInput, so I'd recommend you take a look into the
HttpStaticFileServerHandler.

> Other option that i tried was jetty with continuation or cometd but i am
> more inclined about netty (since i want to replace my mina udp layer with
> mina and maintain one single library ).
>
> Any help will be appreciated, I have my machine available via webex to debug
> as well.
>
>
> my server gets block at second request in the following code :
>
> 1) in my http server:
> ServerBootstrap bootstrap = new ServerBootstrap(new
> NioServerSocketChannelFactory(Executors.newCachedThreadPool(),Executors.newCachedThreadPool()));
> bootstrap.setPipelineFactory(new
> HttpServerPipelineFactory(requestHandler));
> bootstrap.bind(new InetSocketAddress(8080));
> 2) in my pipelinefactory:
> public ChannelPipeline getPipeline() throws Exception {
> ChannelPipeline pipeline = pipeline();
> pipeline.addLast("decoder", new HttpRequestDecoder());
> pipeline.addLast("encoder", new HttpResponseEncoder());
> pipeline.addLast("chunkedWriter", new ChunkedWriteHandler());
> pipeline.addLast("handler", requestHandler);
> return pipeline;
> 3)my http request handler sends event using following code
> private void writeResponse(MessageEvent e) {
> ChannelBuffer buf =
> ChannelBuffers.copiedBuffer(responseContent.toString(), "UTF-8");
> buf.setBytes(0, responseContent.toString().getBytes() );
> responseContent.setLength(0);
> boolean close = HttpHeaders.Values.CLOSE.equalsIgnoreCase(request
> .getHeader(HttpHeaders.Names.CONNECTION))
> || request.getProtocolVersion().equals(HttpVersion.HTTP_1_0)
> && !HttpHeaders.Values.KEEP_ALIVE.equalsIgnoreCase(request
> .getHeader(HttpHeaders.Names.CONNECTION));
>
> HttpResponse response = new DefaultHttpResponse(HttpVersion.HTTP_1_1,
> HttpResponseStatus.OK);
> //response.setContent(buf);
> //response.setHeader(HttpHeaders.Names.CONTENT_TYPE, "text/html;
> charset=UTF-8");
> response.setHeader(HttpHeaders.Names.CONTENT_TYPE,
> "multipart/x-mixed-replace;boundary=XXoXoX");
> //if (!close)
> {
> response.setHeader("Connection", "Keep-Alive");
> response.setHeader("Cache-Control", "no-cache");
> response.setHeader("Access-Control-Allow-Origin", "*");
> }
>
> String cookieString = request.getHeader(HttpHeaders.Names.COOKIE);
> if (cookieString != null) {
> CookieDecoder cookieDecoder = new CookieDecoder();
> Set<Cookie> cookies = cookieDecoder.decode(cookieString);
> if (!cookies.isEmpty()) {
> CookieEncoder cookieEncoder = new CookieEncoder(true);
> for (Cookie cookie : cookies) {
> cookieEncoder.addCookie(cookie);
> }
> response.addHeader(HttpHeaders.Names.SET_COOKIE,
> cookieEncoder.encode());
> }
> }
> ChannelFuture future =null;
> future = e.getChannel().write(response);
> String temp ="";
>
> if(future.isSuccess()==false)
> return;
>
> e.getChannel().write(new ChunkedInput() {
>
> int i = 0;
> public Object nextChunk() throws Exception {
> i++;
> String temp ="";
> //temp ="--XXoXoX\r\n";
> temp = "Content-Type: text/plain \r\n\r\n";
> temp += "{\"data\":\""+i+"\"}";
> temp +="\r\n--XXoXoX\r\n";
> ChannelBuffer buf2 = ChannelBuffers.copiedBuffer(temp.toString(),
> "UTF-8");
> Thread.sleep(100);
> return buf2;
> }
>
> @Override
> public boolean hasNextChunk() throws Exception {
> return true;
> }
>
> @Override
> public void close() throws Exception {
> // TODO Auto-generated method stub
>
> }
> });
>
> if(close){
> future.addListener(ChannelFutureListener.CLOSE);
> }
> }
>
> --
> View this message in context: http://n2.nabble.com/How-to-implement-Streaming-server-like-The-Bayeux-Protocol-tp4055722p4055722.html
> Sent from the Netty User Group mailing list archive at Nabble.com.
> _______________________________________________
> netty-users mailing list
> netty-users@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users
>

— Trustin Lee, http://gleamynode.net/

_______________________________________________
netty-users mailing list
netty-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/netty-users

Re: How to implement Streaming server like The Bayeux Protocol

2009-11-24T06:49:13Z

Hi Trustin, i replaced the line in the static file server example:
//ChannelFuture writeFuture = ch.write(new ChunkedFile(raf, 0, fileLength, 8192));
with line :
ChannelFuture writeFuture = e.getChannel().write(new ChunkedInput() {

my server still getting blocked after two requests all other requests get piled up.
What i did:
1) Took the example httpstatic file server
2) Instead of using chunked file use my own chunked input
3) Opened three different browsers: firefox, chrmoe,ie ( to make sure the test does not gets affected by browser)
4) First two requests succeed and data starts streaming to both firefox and chrome but ie request does not event reach my handler.
5) When i escape or close one of the browser the third request goes through and my static file server handler gets the response.

I still get the same results the server gets blocked.

Trustin Lee wrote:

On Tue, Nov 24, 2009 at 1:02 PM, sumitatgharkikhoj
<mailtosumitsingh@gmail.com> wrote:
>
> First of all thank you for the nice framework, forum and documentation.
> My problem is i am trying to write a http server with small footprint ( will
> go as a component in bigger framework) but it needs to have streaming
> support .
> Different clients will connect at different times ranging from 100-1000
> clients max and this server will be used for both streaming and short data.
> the data is streamed using XHR multipart request and continously (24/7) .
> But when i used the example code (please see below) the server blocks after
> first thread.

In your handler, you check if the returned future.isSuccess() is true
or false right after calling write(). It will be almost always false
because the write operation is asynchronous. You are using
ChunkedInput, so I'd recommend you take a look into the
HttpStaticFileServerHandler.

> Other option that i tried was jetty with continuation or cometd but i am
> more inclined about netty (since i want to replace my mina udp layer with
> mina and maintain one single library ).
>
> Any help will be appreciated, I have my machine available via webex to debug
> as well.
>
>
> my server gets block at second request in the following code :
>
> 1) in my http server:
> ServerBootstrap bootstrap = new ServerBootstrap(new
> NioServerSocketChannelFactory(Executors.newCachedThreadPool(),Executors.newCachedThreadPool()));
> bootstrap.setPipelineFactory(new
> HttpServerPipelineFactory(requestHandler));
> bootstrap.bind(new InetSocketAddress(8080));
> 2) in my pipelinefactory:
> public ChannelPipeline getPipeline() throws Exception {
> ChannelPipeline pipeline = pipeline();
> pipeline.addLast("decoder", new HttpRequestDecoder());
> pipeline.addLast("encoder", new HttpResponseEncoder());
> pipeline.addLast("chunkedWriter", new ChunkedWriteHandler());
> pipeline.addLast("handler", requestHandler);
> return pipeline;
> 3)my http request handler sends event using following code
> private void writeResponse(MessageEvent e) {
> ChannelBuffer buf =
> ChannelBuffers.copiedBuffer(responseContent.toString(), "UTF-8");
> buf.setBytes(0, responseContent.toString().getBytes() );
> responseContent.setLength(0);
> boolean close = HttpHeaders.Values.CLOSE.equalsIgnoreCase(request
> .getHeader(HttpHeaders.Names.CONNECTION))
> || request.getProtocolVersion().equals(HttpVersion.HTTP_1_0)
> && !HttpHeaders.Values.KEEP_ALIVE.equalsIgnoreCase(request
> .getHeader(HttpHeaders.Names.CONNECTION));
>
> HttpResponse response = new DefaultHttpResponse(HttpVersion.HTTP_1_1,
> HttpResponseStatus.OK);
> //response.setContent(buf);
> //response.setHeader(HttpHeaders.Names.CONTENT_TYPE, "text/html;
> charset=UTF-8");
> response.setHeader(HttpHeaders.Names.CONTENT_TYPE,
> "multipart/x-mixed-replace;boundary=XXoXoX");
> //if (!close)
> {
> response.setHeader("Connection", "Keep-Alive");
> response.setHeader("Cache-Control", "no-cache");
> response.setHeader("Access-Control-Allow-Origin", "*");
> }
>
> String cookieString = request.getHeader(HttpHeaders.Names.COOKIE);
> if (cookieString != null) {
> CookieDecoder cookieDecoder = new CookieDecoder();
> Set<Cookie> cookies = cookieDecoder.decode(cookieString);
> if (!cookies.isEmpty()) {
> CookieEncoder cookieEncoder = new CookieEncoder(true);
> for (Cookie cookie : cookies) {
> cookieEncoder.addCookie(cookie);
> }
> response.addHeader(HttpHeaders.Names.SET_COOKIE,
> cookieEncoder.encode());
> }
> }
> ChannelFuture future =null;
> future = e.getChannel().write(response);
> String temp ="";
>
> if(future.isSuccess()==false)
> return;
>
> e.getChannel().write(new ChunkedInput() {
>
> int i = 0;
> public Object nextChunk() throws Exception {
> i++;
> String temp ="";
> //temp ="--XXoXoX\r\n";
> temp = "Content-Type: text/plain \r\n\r\n";
> temp += "{\"data\":\""+i+"\"}";
> temp +="\r\n--XXoXoX\r\n";
> ChannelBuffer buf2 = ChannelBuffers.copiedBuffer(temp.toString(),
> "UTF-8");
> Thread.sleep(100);
> return buf2;
> }
>
> @Override
> public boolean hasNextChunk() throws Exception {
> return true;
> }
>
> @Override
> public void close() throws Exception {
> // TODO Auto-generated method stub
>
> }
> });
>
> if(close){
> future.addListener(ChannelFutureListener.CLOSE);
> }
> }
>
> --
> View this message in context: http://n2.nabble.com/How-to-implement-Streaming-server-like-The-Bayeux-Protocol-tp4055722p4055722.html
> Sent from the Netty User Group mailing list archive at Nabble.com.
> _______________________________________________
> netty-users mailing list
> netty-users@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users
>

— Trustin Lee, http://gleamynode.net/

_______________________________________________
netty-users mailing list
netty-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/netty-users

Re: How to implement Streaming server like The Bayeux Protocol

2009-11-23T20:18:11Z

On Tue, Nov 24, 2009 at 1:02 PM, sumitatgharkikhoj
<[hidden email]> wrote:

>
> First of all thank you for the nice framework, forum and documentation.
> My problem is i am trying to write a http server with small footprint ( will
> go as a component in bigger framework) but it needs to have streaming
> support .
> Different clients will connect at different times ranging from 100-1000
> clients max and this server will be used for both streaming and short data.
> the data is streamed using XHR multipart request and continously (24/7) .
> But when i used the example code (please see below) the server blocks after
> first thread.

In your handler, you check if the returned future.isSuccess() is true
or false right after calling write(). It will be almost always false
because the write operation is asynchronous. You are using
ChunkedInput, so I'd recommend you take a look into the
HttpStaticFileServerHandler.

> Other option that i tried was jetty with continuation or cometd but i am
> more inclined about netty (since i want to replace my mina udp layer with
> mina and maintain one single library ).
>
> Any help will be appreciated, I have my machine available via webex to debug
> as well.
>
>
> my server gets block at second request in the following code :
>
> 1) in my http server:
> ServerBootstrap bootstrap = new ServerBootstrap(new
> NioServerSocketChannelFactory(Executors.newCachedThreadPool(),Executors.newCachedThreadPool()));
> bootstrap.setPipelineFactory(new
> HttpServerPipelineFactory(requestHandler));
> bootstrap.bind(new InetSocketAddress(8080));
> 2) in my pipelinefactory:
> public ChannelPipeline getPipeline() throws Exception {
> ChannelPipeline pipeline = pipeline();
> pipeline.addLast("decoder", new HttpRequestDecoder());
> pipeline.addLast("encoder", new HttpResponseEncoder());
> pipeline.addLast("chunkedWriter", new ChunkedWriteHandler());
> pipeline.addLast("handler", requestHandler);
> return pipeline;
> 3)my http request handler sends event using following code
> private void writeResponse(MessageEvent e) {
> ChannelBuffer buf =
> ChannelBuffers.copiedBuffer(responseContent.toString(), "UTF-8");
> buf.setBytes(0, responseContent.toString().getBytes() );
> responseContent.setLength(0);
> boolean close = HttpHeaders.Values.CLOSE.equalsIgnoreCase(request
> .getHeader(HttpHeaders.Names.CONNECTION))
> || request.getProtocolVersion().equals(HttpVersion.HTTP_1_0)
> && !HttpHeaders.Values.KEEP_ALIVE.equalsIgnoreCase(request
> .getHeader(HttpHeaders.Names.CONNECTION));
>
> HttpResponse response = new DefaultHttpResponse(HttpVersion.HTTP_1_1,
> HttpResponseStatus.OK);
> //response.setContent(buf);
> //response.setHeader(HttpHeaders.Names.CONTENT_TYPE, "text/html;
> charset=UTF-8");
> response.setHeader(HttpHeaders.Names.CONTENT_TYPE,
> "multipart/x-mixed-replace;boundary=XXoXoX");
> //if (!close)
> {
> response.setHeader("Connection", "Keep-Alive");
> response.setHeader("Cache-Control", "no-cache");
> response.setHeader("Access-Control-Allow-Origin", "*");
> }
>
> String cookieString = request.getHeader(HttpHeaders.Names.COOKIE);
> if (cookieString != null) {
> CookieDecoder cookieDecoder = new CookieDecoder();
> Set<Cookie> cookies = cookieDecoder.decode(cookieString);
> if (!cookies.isEmpty()) {
> CookieEncoder cookieEncoder = new CookieEncoder(true);
> for (Cookie cookie : cookies) {
> cookieEncoder.addCookie(cookie);
> }
> response.addHeader(HttpHeaders.Names.SET_COOKIE,
> cookieEncoder.encode());
> }
> }
> ChannelFuture future =null;
> future = e.getChannel().write(response);
> String temp ="";
>
> if(future.isSuccess()==false)
> return;
>
> e.getChannel().write(new ChunkedInput() {
>
> int i = 0;
> public Object nextChunk() throws Exception {
> i++;
> String temp ="";
> //temp ="--XXoXoX\r\n";
> temp = "Content-Type: text/plain \r\n\r\n";
> temp += "{\"data\":\""+i+"\"}";
> temp +="\r\n--XXoXoX\r\n";
> ChannelBuffer buf2 = ChannelBuffers.copiedBuffer(temp.toString(),
> "UTF-8");
> Thread.sleep(100);
> return buf2;
> }
>
> @Override
> public boolean hasNextChunk() throws Exception {
> return true;
> }
>
> @Override
> public void close() throws Exception {
> // TODO Auto-generated method stub
>
> }
> });
>
> if(close){
> future.addListener(ChannelFutureListener.CLOSE);
> }
> }
>
> --
> View this message in context: http://n2.nabble.com/How-to-implement-Streaming-server-like-The-Bayeux-Protocol-tp4055722p4055722.html
> Sent from the Netty User Group mailing list archive at Nabble.com.
> _______________________________________________
> netty-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/netty-users
>

— Trustin Lee, http://gleamynode.net/

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

— Trustin Lee, http://gleamynode.net/

How to implement Streaming server like The Bayeux Protocol

2009-11-23T20:02:47Z

First of all thank you for the nice framework, forum and documentation.
My problem is i am trying to write a http server with small footprint ( will go as a component in bigger framework) but it needs to have streaming support .
Different clients will connect at different times ranging from 100-1000 clients max and this server will be used for both streaming and short data.
the data is streamed using XHR multipart request and continously (24/7) .
But when i used the example code (please see below) the server blocks after first thread.
Other option that i tried was jetty with continuation or cometd but i am more inclined about netty (since i want to replace my mina udp layer with mina and maintain one single library ).

Any help will be appreciated, I have my machine available via webex to debug as well.

my server gets block at second request in the following code :

1) in my http server:
ServerBootstrap bootstrap = new ServerBootstrap(new NioServerSocketChannelFactory(Executors.newCachedThreadPool(),Executors.newCachedThreadPool()));
bootstrap.setPipelineFactory(new HttpServerPipelineFactory(requestHandler));
bootstrap.bind(new InetSocketAddress(8080));
2) in my pipelinefactory:
public ChannelPipeline getPipeline() throws Exception {
ChannelPipeline pipeline = pipeline();
pipeline.addLast("decoder", new HttpRequestDecoder());
pipeline.addLast("encoder", new HttpResponseEncoder());
pipeline.addLast("chunkedWriter", new ChunkedWriteHandler());
pipeline.addLast("handler", requestHandler);
return pipeline;
3)my http request handler sends event using following code
private void writeResponse(MessageEvent e) {
ChannelBuffer buf = ChannelBuffers.copiedBuffer(responseContent.toString(), "UTF-8");
buf.setBytes(0, responseContent.toString().getBytes() );
responseContent.setLength(0);
boolean close = HttpHeaders.Values.CLOSE.equalsIgnoreCase(request
.getHeader(HttpHeaders.Names.CONNECTION))
|| request.getProtocolVersion().equals(HttpVersion.HTTP_1_0)
&& !HttpHeaders.Values.KEEP_ALIVE.equalsIgnoreCase(request
.getHeader(HttpHeaders.Names.CONNECTION));

HttpResponse response = new DefaultHttpResponse(HttpVersion.HTTP_1_1, HttpResponseStatus.OK);
//response.setContent(buf);
//response.setHeader(HttpHeaders.Names.CONTENT_TYPE, "text/html; charset=UTF-8");
response.setHeader(HttpHeaders.Names.CONTENT_TYPE, "multipart/x-mixed-replace;boundary=XXoXoX");
//if (!close)
{
response.setHeader("Connection", "Keep-Alive");
response.setHeader("Cache-Control", "no-cache");
response.setHeader("Access-Control-Allow-Origin", "*");
}

String cookieString = request.getHeader(HttpHeaders.Names.COOKIE);
if (cookieString != null) {
CookieDecoder cookieDecoder = new CookieDecoder();
Set<Cookie> cookies = cookieDecoder.decode(cookieString);
if (!cookies.isEmpty()) {
CookieEncoder cookieEncoder = new CookieEncoder(true);
for (Cookie cookie : cookies) {
cookieEncoder.addCookie(cookie);
}
response.addHeader(HttpHeaders.Names.SET_COOKIE, cookieEncoder.encode());
}
}
ChannelFuture future =null;
future = e.getChannel().write(response);
String temp ="";

if(future.isSuccess()==false)
return;

e.getChannel().write(new ChunkedInput() {

int i = 0;
public Object nextChunk() throws Exception {
i++;
String temp ="";
//temp ="--XXoXoX\r\n";
temp = "Content-Type: text/plain \r\n\r\n";
temp += "{\"data\":\""+i+"\"}";
temp +="\r\n--XXoXoX\r\n";
ChannelBuffer buf2 = ChannelBuffers.copiedBuffer(temp.toString(), "UTF-8");
Thread.sleep(100);
return buf2;
}

@Override
public boolean hasNextChunk() throws Exception {
return true;
}

@Override
public void close() throws Exception {
// TODO Auto-generated method stub

}
});

if(close){
future.addListener(ChannelFutureListener.CLOSE);
}
}

Announcing Flazr - RTMP streaming server based on Netty

2009-11-23T11:41:27Z

Hi,

Just a quick message to this list, apologies for the shameless plug ;) A big thanks to Trustin and all others involved for this awesome framework.

I ported the code at http://flazr.com from Apache Mina to Netty in around a month (not full time). Along the way, mainly because I was pretty impressed by Netty, the documentation quality and the way things just fell into place after implementing the basic encoder / decoder - what was originally envisaged as a pure client-side project extended smoothly into a server-side one as well. There's even an experimental proxy server for RTMP in the distribution which was written in a matter of minutes - but is proving very useful for watching the wire and debugging stuff. I hope to write a detailed summary or blog post later, for now hope this message finds people on this list who may be interested in the RTMP protocol - one which used to be somewhat obscure until recently. Hoping to have more updates on the status of this project in the future.

Regards,

Peter.

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

Re: Can you share ChannelFactory / worker threads in a p2p app?

2009-11-22T17:38:14Z

There is probably not much of a performance difference looking at the individual app. I want to be able to run a large number of instances on a single machine (for testing), and number of threads might become an issue in this case.

I've put a simple (as of yet untested) suggestion together, as for how worker sharing might be archived. I noticed that NioServerSocketPipelineSink and NioClientSocketPipelineSink implements the same round-robin load balancing over a worker array, and that the sinks make no use of workerExecutor and workerCount once the worker array has been constructed. I've factored this out into a NioWorkerPool class, and changed the sinks to receive this instead. The channel factories have had a new constructor added, that takes a pre-constructed workerPool, thereby making it possible to share the pool between clients and servers.

Have included the code below (as a patch against trunk). A few issues: Need to assign a useful bossID to a shared worker, and also change NioWorker so that an appropriate threadName is selected. Do you foresee any issues with this solution?

Regards

Jannick Bitsch

### Eclipse Workspace Patch 1.0

#P Netty

Index: src/org/jboss/netty/channel/socket/nio/NioClientSocketChannelFactory.java

===================================================================

--- src/org/jboss/netty/channel/socket/nio/NioClientSocketChannelFactory.java (revision 1904)

+++ src/org/jboss/netty/channel/socket/nio/NioClientSocketChannelFactory.java (working copy)

@@ -118,23 +118,30 @@

public NioClientSocketChannelFactory(

Executor bossExecutor, Executor workerExecutor,

int workerCount) {

+ this(bossExecutor, new NioWorkerPool(workerExecutor, workerCount));

+ }

+ /**

+ * Creates a new instance.

+ *

+ * @param bossExecutor

+ * the {@link Executor} which will execute the boss thread

+ * @param workerPool

+ * the {@link NioWorkerPool} which will supply nio workers

+ */

+ public NioClientSocketChannelFactory(

+ Executor bossExecutor, NioWorkerPool workerPool) {

if (bossExecutor == null) {

throw new NullPointerException("bossExecutor");

}

- if (workerExecutor == null) {

- throw new NullPointerException("workerExecutor");

- }

- if (workerCount <= 0) {

- throw new IllegalArgumentException(

- "workerCount (" + workerCount + ") " +

- "must be a positive integer.");

- }

this.bossExecutor = bossExecutor;

- this.workerExecutor = workerExecutor;

- sink = new NioClientSocketPipelineSink(bossExecutor, workerExecutor, workerCount);

- }

+ this.workerExecutor = workerPool.workerExecutor;

+ sink = new NioClientSocketPipelineSink(bossExecutor, workerPool);

+ }

public SocketChannel newChannel(ChannelPipeline pipeline) {

return new NioClientSocketChannel(this, pipeline, sink, sink.nextWorker());

}

Index: src/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink.java

===================================================================

--- src/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink.java (revision 1904)

+++ src/org/jboss/netty/channel/socket/nio/NioClientSocketPipelineSink.java (working copy)

@@ -63,16 +63,12 @@

final int id = nextId.incrementAndGet();

final Executor bossExecutor;

private final Boss boss = new Boss();

- private final NioWorker[] workers;

- private final AtomicInteger workerIndex = new AtomicInteger();

+ private final NioWorkerPool workerPool;

NioClientSocketPipelineSink(

- Executor bossExecutor, Executor workerExecutor, int workerCount) {

+ Executor bossExecutor, NioWorkerPool workerPool) {

this.bossExecutor = bossExecutor;

- workers = new NioWorker[workerCount];

- for (int i = 0; i < workers.length; i ++) {

- workers[i] = new NioWorker(id, i + 1, workerExecutor);

- }

+ this.workerPool = workerPool;

}

public void eventSunk(

@@ -160,8 +156,7 @@

}

NioWorker nextWorker() {

- return workers[Math.abs(

- workerIndex.getAndIncrement() % workers.length)];

+ return workerPool.nextWorker();

}

private final class Boss implements Runnable {

Index: src/org/jboss/netty/channel/socket/nio/NioServerSocketPipelineSink.java

===================================================================

--- src/org/jboss/netty/channel/socket/nio/NioServerSocketPipelineSink.java (revision 1904)

+++ src/org/jboss/netty/channel/socket/nio/NioServerSocketPipelineSink.java (working copy)

@@ -57,14 +57,10 @@

private static final AtomicInteger nextId = new AtomicInteger();

private final int id = nextId.incrementAndGet();

- private final NioWorker[] workers;

- private final AtomicInteger workerIndex = new AtomicInteger();

+ private final NioWorkerPool workerPool;

- NioServerSocketPipelineSink(Executor workerExecutor, int workerCount) {

- workers = new NioWorker[workerCount];

- for (int i = 0; i < workers.length; i ++) {

- workers[i] = new NioWorker(id, i + 1, workerExecutor);

- }

+ NioServerSocketPipelineSink(NioWorkerPool workerPool) {

+ this.workerPool = workerPool;

}

public void eventSunk(

@@ -191,8 +187,7 @@

}

NioWorker nextWorker() {

- return workers[Math.abs(

- workerIndex.getAndIncrement() % workers.length)];

+ return workerPool.nextWorker();

}

private final class Boss implements Runnable {

Index: src/org/jboss/netty/channel/socket/nio/NioWorkerPool.java

===================================================================

--- src/org/jboss/netty/channel/socket/nio/NioWorkerPool.java (revision 0)

+++ src/org/jboss/netty/channel/socket/nio/NioWorkerPool.java (revision 0)

@@ -0,0 +1,34 @@

+package org.jboss.netty.channel.socket.nio;

+import java.util.concurrent.Executor;

+import java.util.concurrent.atomic.AtomicInteger;

+public class NioWorkerPool {

+ private final NioWorker[] workers;

+ private final AtomicInteger workerIndex = new AtomicInteger();

+ final Executor workerExecutor;

+ public NioWorkerPool(Executor workerExecutor, int workerCount){

+ if (workerExecutor == null) {

+ throw new NullPointerException("workerExecutor");

+ }

+ if (workerCount <= 0) {

+ throw new IllegalArgumentException(

+ "workerCount (" + workerCount + ") " +

+ "must be a positive integer.");

+ }

+ this.workerExecutor = workerExecutor;

+ workers = new NioWorker[workerCount];

+ for (int i = 0; i < workers.length; i ++) {

+ workers[i] = new NioWorker(0xBEEF, i + 1, workerExecutor);

+ }

+ public NioWorker nextWorker() {

+ return workers[Math.abs(

+ workerIndex.getAndIncrement() % workers.length)];

+ }

\ No newline at end of file

Index: src/org/jboss/netty/channel/socket/nio/NioServerSocketChannelFactory.java

===================================================================

--- src/org/jboss/netty/channel/socket/nio/NioServerSocketChannelFactory.java (revision 1904)

+++ src/org/jboss/netty/channel/socket/nio/NioServerSocketChannelFactory.java (working copy)

@@ -122,22 +122,34 @@

public NioServerSocketChannelFactory(

Executor bossExecutor, Executor workerExecutor,

int workerCount) {

+ this(bossExecutor, new NioWorkerPool(workerExecutor, workerCount));

+ }

+ /**

+ * Creates a new instance.

+ *

+ * @param bossExecutor

+ * the {@link Executor} which will execute the boss threads

+ * @param workerPool

+ * the {@link NioWorkerPool} which will supply nio workers

+ */

+ public NioServerSocketChannelFactory(

+ Executor bossExecutor, NioWorkerPool workerPool){

if (bossExecutor == null) {

throw new NullPointerException("bossExecutor");

}

- if (workerExecutor == null) {

- throw new NullPointerException("workerExecutor");

+ if (workerPool == null) {

+ throw new NullPointerException("workerPool");

}

- if (workerCount <= 0) {

- throw new IllegalArgumentException(

- "workerCount (" + workerCount + ") " +

- "must be a positive integer.");

- }

- this.bossExecutor = bossExecutor;

- this.workerExecutor = workerExecutor;

- sink = new NioServerSocketPipelineSink(workerExecutor, workerCount);

+ this.bossExecutor = bossExecutor;

+ this.workerExecutor = workerPool.workerExecutor;

+ sink = new NioServerSocketPipelineSink(workerPool);

}

public ServerSocketChannel newChannel(ChannelPipeline pipeline) {

return new NioServerSocketChannel(this, pipeline, sink);

}

2009/11/23 Trustin Lee [via Netty Forums and Mailing Lists] <[hidden email]>

When I design Netty API, sharing worker threads for client and server
side seemed to complicate the API too much comparing to the gained
performance, and therefore the worker threads are not shared by
different ChannelFactories.

If there is obvious advantage in performance, I'd like to implement it
though. Please let me know if you find something.

Thanks

— Trustin Lee, http://gleamynode.net/

On Mon, Nov 23, 2009 at 8:42 AM, Thingfish <[hidden email]> wrote:

>
> Hi
>
> I'm currently in the process of coding a peer 2 peer app on top of Netty. By
> definition each peer acts as both a server and a client, which means that I
> currently make use of both the ClientBootstrap and ServerBootstrap to
> support "connection setup" in both directions. Once established "incomming"
> and "outgoing" connections are indistinct able, and for this reason they
> share the same pipeline factory.
>
> My question is, if its possible to also have them share the same worker
> threads and selectors? Currently the use of seperate NioClient and NioServer
> SocketChannelFactories means, that two disjoint sets of worker threads and
> I/O resources are created. As both these worker pools deliver the exact same
> service, it would be more efficient to avoid the extra threads.
>
> --
> View this message in context: http://n2.nabble.com/Can-you-share-ChannelFactory-worker-threads-in-a-p2p-app-tp4048241p4048241.html
> Sent from the Netty User Group mailing list archive at Nabble.com.
> _______________________________________________
> netty-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/netty-users
>

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users
— Trustin Lee, http://gleamynode.net/

Re: Can you share ChannelFactory / worker threads in a p2p app?

2009-11-22T16:10:11Z

When I design Netty API, sharing worker threads for client and server
side seemed to complicate the API too much comparing to the gained
performance, and therefore the worker threads are not shared by
different ChannelFactories.

If there is obvious advantage in performance, I'd like to implement it
though. Please let me know if you find something.

Thanks

— Trustin Lee, http://gleamynode.net/

On Mon, Nov 23, 2009 at 8:42 AM, Thingfish <[hidden email]> wrote:

>
> Hi
>
> I'm currently in the process of coding a peer 2 peer app on top of Netty. By
> definition each peer acts as both a server and a client, which means that I
> currently make use of both the ClientBootstrap and ServerBootstrap to
> support "connection setup" in both directions. Once established "incomming"
> and "outgoing" connections are indistinct able, and for this reason they
> share the same pipeline factory.
>
> My question is, if its possible to also have them share the same worker
> threads and selectors? Currently the use of seperate NioClient and NioServer
> SocketChannelFactories means, that two disjoint sets of worker threads and
> I/O resources are created. As both these worker pools deliver the exact same
> service, it would be more efficient to avoid the extra threads.
>
> --
> View this message in context: http://n2.nabble.com/Can-you-share-ChannelFactory-worker-threads-in-a-p2p-app-tp4048241p4048241.html
> Sent from the Netty User Group mailing list archive at Nabble.com.
> _______________________________________________
> netty-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/netty-users
>

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

— Trustin Lee, http://gleamynode.net/

Can you share ChannelFactory / worker threads in a p2p app?

2009-11-22T15:42:47Z

Hi

I'm currently in the process of coding a peer 2 peer app on top of Netty. By definition each peer acts as both a server and a client, which means that I currently make use of both the ClientBootstrap and ServerBootstrap to support "connection setup" in both directions. Once established "incomming" and "outgoing" connections are indistinct able, and for this reason they share the same pipeline factory.

My question is, if its possible to also have them share the same worker threads and selectors? Currently the use of seperate NioClient and NioServer SocketChannelFactories means, that two disjoint sets of worker threads and I/O resources are created. As both these worker pools deliver the exact same service, it would be more efficient to avoid the extra threads.

Re: Not receiving all data after attempting to close the channel.

2009-11-20T11:50:14Z

Exactly, Stephen. They are apples and oranges, if not apples and
onions.

Mike

On Nov 20, 2009, at 11:27 AM, Stephen Pape wrote:

>
> I just thought I'd mention that I don't have any problems, with the
> exact
> same code, when I switch to the Oio factory. I don't think that's
> surprising, since writes are blocking in this case.
>
> I still haven't been able to address the issue when using Nio.
>
> -Stephen
>
>
> Stephen Pape wrote:
>>
>> Hello,
>>
>>
>>
>> I'm having a problem with both 3.2.0.ALPHA1 and 3.1.5.GA (haven't
>> tested
>> other versions).
>>
>>
>>
>> Basically, in order to make sure that all pending data is written,
>> I have
>> something like this:
>>
>>
>>
>> private volatile ChannelFuture writeFuture = null;
>>
>> private volatile Channel channel = null; // Set later
>>
>>
>>
>> public void write(Object obj)
>> {
>>
>> writeFuture = channel.write(obj);
>> }
>>
>>
>>
>> public void close()
>>
>> {
>>
>> If(this.writeFuture != null)
>>
>> this.writeFuture.addListener(this);
>>
>> else
>>
>> this.channel.close();
>>
>> }
>>
>>
>>
>> public void operationComplete(ChannelFuture future)
>>
>> {
>>
>> // Thread.sleep(1000);
>>
>> future.getChannel.close();
>>
>> }
>>
>>
>>
>>
>>
>> This class is being used to write Objects in a loop. Immediately
>> after the
>> loop the close method is called.
>>
>>
>>
>> I verified that the correct writeFuture is being used in the close()
>> method,
>> but the remote side is not receiving all of the data. When I write 10
>> objects across, sometimes only 7 or 8 will make it through.
>>
>>
>>
>> If I uncomment Thread.sleep(), all of the data is consistently
>> received by
>> the remote side, so it would seem somehow I'm closing the channel too
>> early.
>> Is there something I'm doing wrong?
>>
>>
>>
>> Thanks!
>>
>>
>>
>> -Stephen
>>
>>
>>
>> _______________________________________________
>> netty-users mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/netty-users
>>
>>
>
> --
> View this message in context: http://n2.nabble.com/Not-receiving-all-data-after-attempting-to-close-the-channel-tp4033482p4039588.html
> Sent from the Netty User Group mailing list archive at Nabble.com.
> _______________________________________________
> netty-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/netty-users

Mike McGrady
Principal Investigator AF081-028 AFRL SBIR
Senior Engineer
Topia Technology, Inc
1.253.720.3365
[hidden email]

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

Re: Not receiving all data after attempting to close the channel.

2009-11-20T11:27:33Z

I just thought I'd mention that I don't have any problems, with the exact same code, when I switch to the Oio factory. I don't think that's surprising, since writes are blocking in this case.

I still haven't been able to address the issue when using Nio.

-Stephen

Stephen Pape wrote:

Hello,

I'm having a problem with both 3.2.0.ALPHA1 and 3.1.5.GA (haven't tested
other versions).

Basically, in order to make sure that all pending data is written, I have
something like this:

private volatile ChannelFuture writeFuture = null;

private volatile Channel channel = null; // Set later

public void write(Object obj)
{

writeFuture = channel.write(obj);
}

public void close()

{

If(this.writeFuture != null)

this.writeFuture.addListener(this);

else

this.channel.close();

}

public void operationComplete(ChannelFuture future)

{

// Thread.sleep(1000);

future.getChannel.close();

}

This class is being used to write Objects in a loop. Immediately after the
loop the close method is called.

I verified that the correct writeFuture is being used in the close() method,
but the remote side is not receiving all of the data. When I write 10
objects across, sometimes only 7 or 8 will make it through.

If I uncomment Thread.sleep(), all of the data is consistently received by
the remote side, so it would seem somehow I'm closing the channel too early.
Is there something I'm doing wrong?

Thanks!

-Stephen

_______________________________________________
netty-users mailing list
netty-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/netty-users

RE: Not receiving all data after attempting to close the channel.

2009-11-23T07:25:07Z

Hi Trustin,

I finally narrowed down the problem to be my own.
I had made soLinger a configurable option, and it was initialized to 0 by default.

For future reference, when it's set to zero:
"
A close() returns immediately. The underlying stack discards any unsent data, and, in the case of connection-oriented protocols such as TCP, sends a RST (reset) to the peer (this is termed a hard or abortive close). All subsequent attempts by the peer's application to read()/recv() data will result in an ECONNRESET.
"

So I'll use a default value of -1 and not call setOption() if the value is still at -1.

Thanks!

-Stephen

-----Original Message-----
From: Trustin Lee [via Netty Forums and Mailing Lists] [mailto:[hidden email]]
Sent: Friday, November 20, 2009 10:26 PM
To: Pape, Stephen R CTR USAF AFMC AFRL/RISE
Subject: Re: Not receiving all data after attempting to close the channel.

Hi Stephen,

On Fri, Nov 20, 2009 at 4:24 AM, Stephen Pape
<[hidden email] <http://n2.nabble.com/user/SendEmail.jtp?type=node&node=4041435&i=0> > wrote:

I think you are closing the connection properly. I never experienced
the same problem, but it worths an investigation just in case. Can
you reproduce the problem with loopback connection? Do you have a
simple application that reproduces the problem?

Thanks
T

_______________________________________________
netty-users mailing list
[hidden email] <http://n2.nabble.com/user/SendEmail.jtp?type=node&node=4041435&i=1>
https://lists.jboss.org/mailman/listinfo/netty-users

— Trustin Lee, http://gleamynode.net/

________________________________

View message @ http://n2.nabble.com/Not-receiving-all-data-after-attempting-to-close-the-channel-tp4033482p4041435.html
To unsubscribe from Re: Not receiving all data after attempting to close the channel., click here <<Link Removed>> .

smime.p7s (6K) Download Attachment

Re: Not receiving all data after attempting to close the channel.

2009-11-20T19:19:45Z

Hi Stephen,

On Fri, Nov 20, 2009 at 4:24 AM, Stephen Pape
<[hidden email]> wrote:

— Trustin Lee, http://gleamynode.net/

RE: Not receiving all data after attempting to close the channel.

2009-11-19T12:13:45Z

I'm not sure what you mean, as you can see I register a callback for when
the write is supposed to be completed. This should not be closing the
channel before it is done writing... (?)

After the write is *completed* (supposedly), I'm immediately closing the
channel. I don't know why there would be a problem doing this, but it's what
I was asking about.

-Stephen

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Michael McGrady
Sent: Thursday, November 19, 2009 3:01 PM
To: Netty Users
Subject: Re: Not receiving all data after attempting to close the channel.

Right. If you close the channel before it is done, guess what? It won't
get done. LOL Hope you take this in the light way it is intended. The
serious point is that you have to not close before you are done.

What the difference is between your version and what Trustin suggested I
don't know. I do think I am correct that you cannot close the channel
before it is complete without buggering up the send.

Mike

On Nov 19, 2009, at 11:24 AM, Stephen Pape wrote:

>
> Thanks for the reply. Isn't what you're doing essentially the same thing?
> You're getting notified via operationComplete() that the write was
> (supposedly) completed, then allowing execution to continue. That seems
like

> what I'm doing, except I immediately close the channel.
>
> I found a similar thread about this topic here:
> http://www.jboss.org/netty/community.html#nabble-td3398725|a3398725
>
>
> Trustin Lee wrote:
>>
>> You can achieve this pretty simply:
>>
>> channel.write(ChannelBuffers.EMPTY_BUFFER)
>> .addListener(ChannelFutureListener.CLOSE);
>>
>
> Given that response, I assumed my method should work. I was doing the same
> thing, adding a listener to close after the last write completed. I wasn't
> using an empty buffer because my channel pipeline couldn't accept a
> ChannelBuffer directly.
>
> Is there some reason that the channel can't be closed immediately after a
> write has completed?
>
> -Stephen
>
>
> Mike McGrady wrote:
>>
>> My solution was to use monitors with the asynchronous Netty

functionality.
>>
>> Here is the monitor:
>>
>>
>>
>> private class WriteKarmaManagerMonitor {
>> private final AtomicBoolean managerIsSent = new

AtomicBoolean(false);

>>
>> protected void managerIsSent() {
>> this.managerIsSent.set(true);
>> }
>>
>> protected AtomicBoolean isManagerSent() {
>> return this.managerIsSent;
>> }
>> }
>>
>> And here is its use:
>>
>>
>>
>> while (!this.channel.isWritable()) {
>> try {
>> Thread.sleep(3);
>> } catch (InterruptedException e) {
>> logger.log(InternalLogLevel.ERROR,

e.getMessage());
>> }
>> }
>>
>> ChannelFuture future = this.channel.write(buffer);
>>
>> future.addListener(new ChannelFutureListener() {
>>
>> public void operationComplete(ChannelFuture future)

{
>> // Notify that snnd is complete monitor.
>> synchronized
(NettyKarmaSender.this.writeManagerMonitor) {
>>
>>
NettyKarmaSender.this.writeManagerMonitor.managerIsSent();
>>
NettyKarmaSender.this.writeManagerMonitor.notifyAll();

>> NettyKarmaSender.this.managerSent++;
>> count++;
>> }
>> }
>>
>> });
>>
>> // Make sure manager is written before proceeding.
>> synchronized (this.writeManagerMonitor) {
>> if (!this.writeManagerMonitor.isManagerSent().get())

{
>> try {
>>
>> this.writeManagerMonitor.wait();
>> } catch (InterruptedException e) {
>> logger.log(InternalLogLevel.ERROR,

e.getMessage());

>> }
>> }
>> }
>>
>> From: "Pape, Stephen R CTR USAF AFMC AFRL/RISE"
>> <[hidden email]>
>> Date: November 19, 2009 7:24:17 AM PST
>> To: <[hidden email]>
>> Subject: Not receiving all data after attempting to close the channel.
>> Reply-To: Netty Users <[hidden email]>
>>
>> Hello,
>>
>> I'm having a problem with both 3.2.0.ALPHA1 and 3.1.5.GA (haven't tested
>> other versions).
>>
>> Basically, in order to make sure that all pending data is written, I have
>> something like this:
>>
>> private volatile ChannelFuture writeFuture = null;
>> private volatile Channel channel = null; // Set later
>>
>> public void write(Object obj)
>> {
>> writeFuture = channel.write(obj);
>> }
>>
>> public void close()
>> {
>> If(this.writeFuture != null)
>> this.writeFuture.addListener(this);
>> else
>> this.channel.close();
>> }
>>
>> public void operationComplete(ChannelFuture future)
>> {
>> // Thread.sleep(1000);
>> future.getChannel.close();
>> }
>>
>>
>> This class is being used to write Objects in a loop. Immediately after

the
>> loop the close method is called.
>>
>> I verified that the correct writeFuture is being used in the close()
>> method, but the remote side is not receiving all of the data. When I

write
>> 10 objects across, sometimes only 7 or 8 will make it through.
>>
>> If I uncomment Thread.sleep(), all of the data is consistently received
by

>> the remote side, so it would seem somehow I'm closing the channel too
>> early. Is there something I'm doing wrong?
>>
>> Thanks!
>>
>> -Stephen
>> _______________________________________________
>> netty-users mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/netty-users
>>
>>
>> From: "Pape, Stephen R CTR USAF AFMC AFRL/RISE"
>> <[hidden email]>
>> Date: November 19, 2009 7:24:17 AM PST
>> To: <[hidden email]>
>> Subject: Not receiving all data after attempting to close the channel.
>> Reply-To: Netty Users <[hidden email]>
>>
>> Hello,
>>
>> I'm having a problem with both 3.2.0.ALPHA1 and 3.1.5.GA (haven't tested
>> other versions).
>>
>> Basically, in order to make sure that all pending data is written, I have
>> something like this:
>>
>> private volatile ChannelFuture writeFuture = null;
>> private volatile Channel channel = null; // Set later
>>
>> public void write(Object obj)
>> {
>> writeFuture = channel.write(obj);
>> }
>>
>> public void close()
>> {
>> If(this.writeFuture != null)
>> this.writeFuture.addListener(this);
>> else
>> this.channel.close();
>> }
>>
>> public void operationComplete(ChannelFuture future)
>> {
>> // Thread.sleep(1000);
>> future.getChannel.close();
>> }
>>
>>
>> This class is being used to write Objects in a loop. Immediately after

the
>> loop the close method is called.
>>
>> I verified that the correct writeFuture is being used in the close()
>> method, but the remote side is not receiving all of the data. When I

write
>> 10 objects across, sometimes only 7 or 8 will make it through.
>>
>> If I uncomment Thread.sleep(), all of the data is consistently received
by

have

>>> something like this:
>>>
>>> private volatile ChannelFuture writeFuture = null;
>>> private volatile Channel channel = null; // Set later
>>>
>>> public void write(Object obj)
>>> {
>>> writeFuture = channel.write(obj);
>>> }
>>>
>>> public void close()
>>> {
>>> If(this.writeFuture != null)
>>> this.writeFuture.addListener(this);
>>> else
>>> this.channel.close();
>>> }
>>>
>>> public void operationComplete(ChannelFuture future)
>>> {
>>> // Thread.sleep(1000);
>>> future.getChannel.close();
>>> }
>>>
>>>
>>> This class is being used to write Objects in a loop. Immediately after
>>> the loop the close method is called.
>>>
>>> I verified that the correct writeFuture is being used in the close()
>>> method, but the remote side is not receiving all of the data. When I
>>> write 10 objects across, sometimes only 7 or 8 will make it through.
>>>
>>> If I uncomment Thread.sleep(), all of the data is consistently received
>>> by the remote side, so it would seem somehow I'm closing the channel too
>>> early. Is there something I'm doing wrong?
>>>
>>> Thanks!
>>>
>>> -Stephen
>>> _______________________________________________
>>> netty-users mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>
>>
>> _______________________________________________
>> netty-users mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/netty-users
>>
>>
>
> --
> View this message in context:

http://n2.nabble.com/Not-receiving-all-data-after-attempting-to-close-the-ch

annel-tp4033482p4033860.html
> Sent from the Netty User Group mailing list archive at Nabble.com.
>
> _______________________________________________
> netty-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/netty-users

Mike McGrady
Principal Investigator AF081-028 AFRL SBIR
Senior Engineer
Topia Technology, Inc
1.253.720.3365
[hidden email]

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

smime.p7s (6K) Download Attachment

Re: Not receiving all data after attempting to close the channel.

2009-11-19T12:00:56Z

Right. If you close the channel before it is done, guess what? It won't get done. LOL Hope you take this in the light way it is intended. The serious point is that you have to not close before you are done.

What the difference is between your version and what Trustin suggested I don't know. I do think I am correct that you cannot close the channel before it is complete without buggering up the send.

Mike

On Nov 19, 2009, at 11:24 AM, Stephen Pape wrote:

>
> Thanks for the reply. Isn't what you're doing essentially the same thing?
> You're getting notified via operationComplete() that the write was
> (supposedly) completed, then allowing execution to continue. That seems like
> what I'm doing, except I immediately close the channel.
>
> I found a similar thread about this topic here:
> http://www.jboss.org/netty/community.html#nabble-td3398725|a3398725
>
>
> Trustin Lee wrote:
>>
>> You can achieve this pretty simply:
>>
>> channel.write(ChannelBuffers.EMPTY_BUFFER)
>> .addListener(ChannelFutureListener.CLOSE);
>>
>
> Given that response, I assumed my method should work. I was doing the same
> thing, adding a listener to close after the last write completed. I wasn't
> using an empty buffer because my channel pipeline couldn't accept a
> ChannelBuffer directly.
>
> Is there some reason that the channel can't be closed immediately after a
> write has completed?
>
> -Stephen
>
>
> Mike McGrady wrote:
>>
>> My solution was to use monitors with the asynchronous Netty functionality.
>>
>> Here is the monitor:
>>
>>
>>
>> private class WriteKarmaManagerMonitor {
>> private final AtomicBoolean managerIsSent = new AtomicBoolean(false);
>>
>> protected void managerIsSent() {
>> this.managerIsSent.set(true);
>> }
>>
>> protected AtomicBoolean isManagerSent() {
>> return this.managerIsSent;
>> }
>> }
>>
>> And here is its use:
>>
>>
>>
>> while (!this.channel.isWritable()) {
>> try {
>> Thread.sleep(3);
>> } catch (InterruptedException e) {
>> logger.log(InternalLogLevel.ERROR, e.getMessage());
>> }
>> }
>>
>> ChannelFuture future = this.channel.write(buffer);
>>
>> future.addListener(new ChannelFutureListener() {
>>
>> public void operationComplete(ChannelFuture future) {
>> // Notify that snnd is complete monitor.
>> synchronized (NettyKarmaSender.this.writeManagerMonitor) {
>>
>> NettyKarmaSender.this.writeManagerMonitor.managerIsSent();
>> NettyKarmaSender.this.writeManagerMonitor.notifyAll();
>> NettyKarmaSender.this.managerSent++;
>> count++;
>> }
>> }
>>
>> });
>>
>> // Make sure manager is written before proceeding.
>> synchronized (this.writeManagerMonitor) {
>> if (!this.writeManagerMonitor.isManagerSent().get()) {
>> try {
>>
>> this.writeManagerMonitor.wait();
>> } catch (InterruptedException e) {
>> logger.log(InternalLogLevel.ERROR, e.getMessage());
>> }
>> }
>> }
>>
>> From: "Pape, Stephen R CTR USAF AFMC AFRL/RISE"
>> <[hidden email]>
>> Date: November 19, 2009 7:24:17 AM PST
>> To: <[hidden email]>
>> Subject: Not receiving all data after attempting to close the channel.
>> Reply-To: Netty Users <[hidden email]>
>>
>> Hello,
>>
>> I’m having a problem with both 3.2.0.ALPHA1 and 3.1.5.GA (haven’t tested
>> other versions).
>>
>> Basically, in order to make sure that all pending data is written, I have
>> something like this:
>>
>> private volatile ChannelFuture writeFuture = null;
>> private volatile Channel channel = null; // Set later
>>
>> public void write(Object obj)
>> {
>> writeFuture = channel.write(obj);
>> }
>>
>> public void close()
>> {
>> If(this.writeFuture != null)
>> this.writeFuture.addListener(this);
>> else
>> this.channel.close();
>> }
>>
>> public void operationComplete(ChannelFuture future)
>> {
>> // Thread.sleep(1000);
>> future.getChannel.close();
>> }
>>
>>
>> This class is being used to write Objects in a loop. Immediately after the
>> loop the close method is called.
>>
>> I verified that the correct writeFuture is being used in the close()
>> method, but the remote side is not receiving all of the data. When I write
>> 10 objects across, sometimes only 7 or 8 will make it through.
>>
>> If I uncomment Thread.sleep(), all of the data is consistently received by
>> the remote side, so it would seem somehow I’m closing the channel too
>> early. Is there something I’m doing wrong?
>>
>> Thanks!
>>
>> -Stephen
>> _______________________________________________
>> netty-users mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/netty-users
>>
>>
>> From: "Pape, Stephen R CTR USAF AFMC AFRL/RISE"
>> <[hidden email]>
>> Date: November 19, 2009 7:24:17 AM PST
>> To: <[hidden email]>
>> Subject: Not receiving all data after attempting to close the channel.
>> Reply-To: Netty Users <[hidden email]>
>>
>> Hello,
>>
>> I’m having a problem with both 3.2.0.ALPHA1 and 3.1.5.GA (haven’t tested
>> other versions).
>>
>> Basically, in order to make sure that all pending data is written, I have
>> something like this:
>>
>> private volatile ChannelFuture writeFuture = null;
>> private volatile Channel channel = null; // Set later
>>
>> public void write(Object obj)
>> {
>> writeFuture = channel.write(obj);
>> }
>>
>> public void close()
>> {
>> If(this.writeFuture != null)
>> this.writeFuture.addListener(this);
>> else
>> this.channel.close();
>> }
>>
>> public void operationComplete(ChannelFuture future)
>> {
>> // Thread.sleep(1000);
>> future.getChannel.close();
>> }
>>
>>
>> This class is being used to write Objects in a loop. Immediately after the
>> loop the close method is called.
>>
>> I verified that the correct writeFuture is being used in the close()
>> method, but the remote side is not receiving all of the data. When I write
>> 10 objects across, sometimes only 7 or 8 will make it through.
>>
>> If I uncomment Thread.sleep(), all of the data is consistently received by
>> the remote side, so it would seem somehow I’m closing the channel too
>> early. Is there something I’m doing wrong?
>>
>> Thanks!
>>
>> -Stephen
>> _______________________________________________
>> netty-users mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/netty-users
>>
>>
>>
>>
>>
>> On Nov 19, 2009, at 7:24 AM, Pape, Stephen R CTR USAF AFMC AFRL/RISE
>> wrote:
>>
>>> Hello,
>>>
>>> I’m having a problem with both 3.2.0.ALPHA1 and 3.1.5.GA (haven’t tested
>>> other versions).
>>>
>>> Basically, in order to make sure that all pending data is written, I have
>>> something like this:
>>>
>>> private volatile ChannelFuture writeFuture = null;
>>> private volatile Channel channel = null; // Set later
>>>
>>> public void write(Object obj)
>>> {
>>> writeFuture = channel.write(obj);
>>> }
>>>
>>> public void close()
>>> {
>>> If(this.writeFuture != null)
>>> this.writeFuture.addListener(this);
>>> else
>>> this.channel.close();
>>> }
>>>
>>> public void operationComplete(ChannelFuture future)
>>> {
>>> // Thread.sleep(1000);
>>> future.getChannel.close();
>>> }
>>>
>>>
>>> This class is being used to write Objects in a loop. Immediately after
>>> the loop the close method is called.
>>>
>>> I verified that the correct writeFuture is being used in the close()
>>> method, but the remote side is not receiving all of the data. When I
>>> write 10 objects across, sometimes only 7 or 8 will make it through.
>>>
>>> If I uncomment Thread.sleep(), all of the data is consistently received
>>> by the remote side, so it would seem somehow I’m closing the channel too
>>> early. Is there something I’m doing wrong?
>>>
>>> Thanks!
>>>
>>> -Stephen
>>> _______________________________________________
>>> netty-users mailing list
>>> [hidden email]
>>> https://lists.jboss.org/mailman/listinfo/netty-users
>>
>>
>> _______________________________________________
>> netty-users mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/netty-users
>>
>>
>
> --
> View this message in context: http://n2.nabble.com/Not-receiving-all-data-after-attempting-to-close-the-channel-tp4033482p4033860.html
> Sent from the Netty User Group mailing list archive at Nabble.com.
>
> _______________________________________________
> netty-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/netty-users

Re: Not receiving all data after attempting to close the channel.

2009-11-19T11:24:43Z

Thanks for the reply. Isn't what you're doing essentially the same thing? You're getting notified via operationComplete() that the write was (supposedly) completed, then allowing execution to continue. That seems like what I'm doing, except I immediately close the channel.

I found a similar thread about this topic here: http://www.jboss.org/netty/community.html#nabble-td3398725|a3398725

Trustin Lee wrote:

You can achieve this pretty simply:

channel.write(ChannelBuffers.EMPTY_BUFFER)
.addListener(ChannelFutureListener.CLOSE);

Given that response, I assumed my method should work. I was doing the same thing, adding a listener to close after the last write completed. I wasn't using an empty buffer because my channel pipeline couldn't accept a ChannelBuffer directly.

Is there some reason that the channel can't be closed immediately after a write has completed?

-Stephen

Mike McGrady wrote:

My solution was to use monitors with the asynchronous Netty functionality.

Here is the monitor:

private class WriteKarmaManagerMonitor {
private final AtomicBoolean managerIsSent = new AtomicBoolean(false);

protected void managerIsSent() {
this.managerIsSent.set(true);
}

protected AtomicBoolean isManagerSent() {
return this.managerIsSent;
}
}

And here is its use:

while (!this.channel.isWritable()) {
try {
Thread.sleep(3);
} catch (InterruptedException e) {
logger.log(InternalLogLevel.ERROR, e.getMessage());
}
}

ChannelFuture future = this.channel.write(buffer);

future.addListener(new ChannelFutureListener() {

public void operationComplete(ChannelFuture future) {
// Notify that snnd is complete monitor.
synchronized (NettyKarmaSender.this.writeManagerMonitor) {

NettyKarmaSender.this.writeManagerMonitor.managerIsSent();
NettyKarmaSender.this.writeManagerMonitor.notifyAll();
NettyKarmaSender.this.managerSent++;
count++;
}
}

});

// Make sure manager is written before proceeding.
synchronized (this.writeManagerMonitor) {
if (!this.writeManagerMonitor.isManagerSent().get()) {
try {

this.writeManagerMonitor.wait();
} catch (InterruptedException e) {
logger.log(InternalLogLevel.ERROR, e.getMessage());
}
}
}

From: "Pape, Stephen R CTR USAF AFMC AFRL/RISE" <Stephen.Pape.ctr@RL.af.mil>
Date: November 19, 2009 7:24:17 AM PST
To: <netty-users@lists.jboss.org>
Subject: Not receiving all data after attempting to close the channel.
Reply-To: Netty Users <netty-users@lists.jboss.org>

Hello,

I’m having a problem with both 3.2.0.ALPHA1 and 3.1.5.GA (haven’t tested other versions).

Basically, in order to make sure that all pending data is written, I have something like this:

private volatile ChannelFuture writeFuture = null;
private volatile Channel channel = null; // Set later

public void write(Object obj)
{
writeFuture = channel.write(obj);
}

public void close()
{
If(this.writeFuture != null)
this.writeFuture.addListener(this);
else
this.channel.close();
}

public void operationComplete(ChannelFuture future)
{
// Thread.sleep(1000);
future.getChannel.close();
}

This class is being used to write Objects in a loop. Immediately after the loop the close method is called.

I verified that the correct writeFuture is being used in the close() method, but the remote side is not receiving all of the data. When I write 10 objects across, sometimes only 7 or 8 will make it through.

If I uncomment Thread.sleep(), all of the data is consistently received by the remote side, so it would seem somehow I’m closing the channel too early. Is there something I’m doing wrong?

Thanks!

-Stephen
_______________________________________________
netty-users mailing list
netty-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/netty-users

From: "Pape, Stephen R CTR USAF AFMC AFRL/RISE" <Stephen.Pape.ctr@RL.af.mil>
Date: November 19, 2009 7:24:17 AM PST
To: <netty-users@lists.jboss.org>
Subject: Not receiving all data after attempting to close the channel.
Reply-To: Netty Users <netty-users@lists.jboss.org>

Hello,

I’m having a problem with both 3.2.0.ALPHA1 and 3.1.5.GA (haven’t tested other versions).

Basically, in order to make sure that all pending data is written, I have something like this:

private volatile ChannelFuture writeFuture = null;
private volatile Channel channel = null; // Set later

public void write(Object obj)
{
writeFuture = channel.write(obj);
}

public void close()
{
If(this.writeFuture != null)
this.writeFuture.addListener(this);
else
this.channel.close();
}

public void operationComplete(ChannelFuture future)
{
// Thread.sleep(1000);
future.getChannel.close();
}

This class is being used to write Objects in a loop. Immediately after the loop the close method is called.

I verified that the correct writeFuture is being used in the close() method, but the remote side is not receiving all of the data. When I write 10 objects across, sometimes only 7 or 8 will make it through.

If I uncomment Thread.sleep(), all of the data is consistently received by the remote side, so it would seem somehow I’m closing the channel too early. Is there something I’m doing wrong?

Thanks!

-Stephen
_______________________________________________
netty-users mailing list
netty-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/netty-users

On Nov 19, 2009, at 7:24 AM, Pape, Stephen R CTR USAF AFMC AFRL/RISE wrote:

> Hello,
>
> I’m having a problem with both 3.2.0.ALPHA1 and 3.1.5.GA (haven’t tested other versions).
>
> Basically, in order to make sure that all pending data is written, I have something like this:
>
> private volatile ChannelFuture writeFuture = null;
> private volatile Channel channel = null; // Set later
>
> public void write(Object obj)
> {
> writeFuture = channel.write(obj);
> }
>
> public void close()
> {
> If(this.writeFuture != null)
> this.writeFuture.addListener(this);
> else
> this.channel.close();
> }
>
> public void operationComplete(ChannelFuture future)
> {
> // Thread.sleep(1000);
> future.getChannel.close();
> }
>
>
> This class is being used to write Objects in a loop. Immediately after the loop the close method is called.
>
> I verified that the correct writeFuture is being used in the close() method, but the remote side is not receiving all of the data. When I write 10 objects across, sometimes only 7 or 8 will make it through.
>
> If I uncomment Thread.sleep(), all of the data is consistently received by the remote side, so it would seem somehow I’m closing the channel too early. Is there something I’m doing wrong?
>
> Thanks!
>
> -Stephen
> _______________________________________________
> netty-users mailing list
> netty-users@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users

_______________________________________________
netty-users mailing list
netty-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/netty-users

Re: Not receiving all data after attempting to close the channel.

2009-11-19T10:59:41Z

My solution was to use monitors with the asynchronous Netty functionality.

Here is the monitor:

private class WriteKarmaManagerMonitor {

private final AtomicBoolean managerIsSent = new AtomicBoolean(false);

protected void managerIsSent() {

this.managerIsSent.set(true);

}

protected AtomicBoolean isManagerSent() {

return this.managerIsSent;

}

And here is its use:

while (!this.channel.isWritable()) {

try {

Thread.sleep(3);

} catch (InterruptedException e) {

logger.log(InternalLogLevel.ERROR, e.getMessage());

}

ChannelFuture future = this.channel.write(buffer);

future.addListener(new ChannelFutureListener() {

public void operationComplete(ChannelFuture future) {

// Notify that snnd is complete monitor.

synchronized (NettyKarmaSender.this.writeManagerMonitor) {

NettyKarmaSender.this.writeManagerMonitor.managerIsSent();

NettyKarmaSender.this.writeManagerMonitor.notifyAll();

NettyKarmaSender.this.managerSent++;

count++;

}

});

// Make sure manager is written before proceeding.

synchronized (this.writeManagerMonitor) {

if (!this.writeManagerMonitor.isManagerSent().get()) {

try {

this.writeManagerMonitor.wait();

} catch (InterruptedException e) {

logger.log(InternalLogLevel.ERROR, e.getMessage());

}

From: "Pape, Stephen R CTR USAF AFMC AFRL/RISE" <[hidden email]>

Date: November 19, 2009 7:24:17 AM PST

To: <[hidden email]>

Subject: Not receiving all data after attempting to close the channel.

Reply-To: Netty Users <[hidden email]>

Hello,

I’m having a problem with both 3.2.0.ALPHA1 and 3.1.5.GA (haven’t tested other versions).

Basically, in order to make sure that all pending data is written, I have something like this:

private volatile ChannelFuture writeFuture = null;

private volatile Channel channel = null; // Set later

public void write(Object obj)
{

writeFuture = channel.write(obj);
}

public void close()

{

If(this.writeFuture != null)

this.writeFuture.addListener(this);

else

this.channel.close();

}

public void operationComplete(ChannelFuture future)

{

// Thread.sleep(1000);

future.getChannel.close();

}

This class is being used to write Objects in a loop. Immediately after the loop the close method is called.

I verified that the correct writeFuture is being used in the close() method, but the remote side is not receiving all of the data. When I write 10 objects across, sometimes only 7 or 8 will make it through.

If I uncomment Thread.sleep(), all of the data is consistently received by the remote side, so it would seem somehow I’m closing the channel too early. Is there something I’m doing wrong?

Thanks!

-Stephen

_______________________________________________

netty-users mailing list

[hidden email]

https://lists.jboss.org/mailman/listinfo/netty-users

From: "Pape, Stephen R CTR USAF AFMC AFRL/RISE" <[hidden email]>

Date: November 19, 2009 7:24:17 AM PST

To: <[hidden email]>

Subject: Not receiving all data after attempting to close the channel.

Reply-To: Netty Users <[hidden email]>

Hello,

I’m having a problem with both 3.2.0.ALPHA1 and 3.1.5.GA (haven’t tested other versions).

Basically, in order to make sure that all pending data is written, I have something like this:

private volatile ChannelFuture writeFuture = null;

private volatile Channel channel = null; // Set later

public void write(Object obj)
{

writeFuture = channel.write(obj);
}

public void close()

{

If(this.writeFuture != null)

this.writeFuture.addListener(this);

else

this.channel.close();

}

public void operationComplete(ChannelFuture future)

{

// Thread.sleep(1000);

future.getChannel.close();

}

This class is being used to write Objects in a loop. Immediately after the loop the close method is called.

If I uncomment Thread.sleep(), all of the data is consistently received by the remote side, so it would seem somehow I’m closing the channel too early. Is there something I’m doing wrong?

Thanks!

-Stephen

_______________________________________________

netty-users mailing list

[hidden email]

https://lists.jboss.org/mailman/listinfo/netty-users

On Nov 19, 2009, at 7:24 AM, Pape, Stephen R CTR USAF AFMC AFRL/RISE wrote:

Hello,

I’m having a problem with both 3.2.0.ALPHA1 and 3.1.5.GA (haven’t tested other versions).

Basically, in order to make sure that all pending data is written, I have something like this:

private volatile ChannelFuture writeFuture = null;
                private volatile Channel channel = null; // Set later

public void write(Object obj)
{
   writeFuture = channel.write(obj);
}

public void close()
{
  If(this.writeFuture != null)
    this.writeFuture.addListener(this);
else
     this.channel.close();
}

public void operationComplete(ChannelFuture future)
{
// Thread.sleep(1000);
future.getChannel.close();
}

This class is being used to write Objects in a loop. Immediately after the loop the close method is called.

I verified that the correct writeFuture is being used in the close() method, but the remote side is not receiving all of the data. When I write 10 objects across, sometimes only 7 or 8 will make it through.

If I uncomment Thread.sleep(), all of the data is consistently received by the remote side, so it would seem somehow I’m closing the channel too early. Is there something I’m doing wrong?

Thanks!

-Stephen
_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

Not receiving all data after attempting to close the channel.

2009-11-19T07:24:17Z

Hello,

I’m having a problem with both 3.2.0.ALPHA1 and 3.1.5.GA (haven’t tested other versions).

Basically, in order to make sure that all pending data is written, I have something like this:

private volatile ChannelFuture writeFuture = null;

private volatile Channel channel = null; // Set later

public void write(Object obj)
{

writeFuture = channel.write(obj);
}

public void close()

{

If(this.writeFuture != null)

this.writeFuture.addListener(this);

else

this.channel.close();

}

public void operationComplete(ChannelFuture future)

{

// Thread.sleep(1000);

future.getChannel.close();

}

This class is being used to write Objects in a loop. Immediately after the loop the close method is called.

If I uncomment Thread.sleep(), all of the data is consistently received by the remote side, so it would seem somehow I’m closing the channel too early. Is there something I’m doing wrong?

Thanks!

-Stephen

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

smime.p7s (6K) Download Attachment

Re: synchronous client send/rcv help

2009-11-20T21:03:36Z

Hi Jason,

I would not suggest using a lock or semaphore. The most efficient way
is to design the state machine carefully so that the premature write
request is not made at all. If it is not easy to implement, you can
store the write requests in the queue while handshake is in progress,
and then flush them all after the handshake.

HTH

— Trustin Lee, http://gleamynode.net/

On Thu, Nov 19, 2009 at 7:24 AM, Jason Ward <[hidden email]> wrote:

> I'm looking for some advice on a good design for the following problem.
>
> I have a need to perform an app-level handshake once the channel is
> connected. I've implemented this via a handshake method call fired from
> the channelConnected(ChannelHandlerContext ctx, ChannelStateEvent e) of
> my IoHandler (extending SimpleChannelUpstreamHandler).
>
> The crux of the issue is a need for this app-level handshake to occur
> synchronously blocking each subsequent send (in the handshake) until a
> previous response from the peer is rcvd and validated. I've found that
> blocking during my handshake method in fact blocks the I/O thread
> altogether. I can see the peer actually receives the req and sends a
> response, but a corresponding messageReceived(ChannelHandlerContext ctx,
> MessageEvent e) isn't fired until after my originating caller times
> out... which just so happens to initiate a disconnect and disposal of
> resources as well.
>
> I can easily workaround this issue by chaining messageReceived
> callbacks, but I'm looking for a more elegant solution. Does anyone have
> advice on how I can pull this off? Is there any example of how to force
> only some requests to be blocking? I've seen the thread on the forum
> where Trustin indicates adding a LinkedBlockingQueue is a solution, but
> the original poster's use-case is quite different from mine in that all
> requests needed to be blocking, and as best I understand it, I'm
> effectively doing the same thing but creating a problem because I'm
> calling my blocking method from channelConnected()... altho I might be
> totally wrong.
>
>
> Any advice is greatly appreciated.
>
>
> Note : Because this might be a simple problem with my I/O thread pools,
> I'll include my client code here (only slightly redacted for protected
> info).
>
> ClientSocketChannelFactory factory = new
> NioClientSocketChannelFactory(
> Executors.newCachedThreadPool(new
> NamedThreadFactory("NioClientSocketChannelFactoryBoss_"+name)),
> Executors.newCachedThreadPool(new
> NamedThreadFactory("NioClientSocketChannelFactory_"+name))
> );
> bootstrap = new ClientBootstrap(factory);
> bootstrap.setOption("tcpNoDelay", true);
> bootstrap.setOption("keepAlive", true);
>
> OrderedMemoryAwareThreadPoolExecutor threadPool = new
> OrderedMemoryAwareThreadPoolExecutor(
> corePoolSize, 0, 0, keepAliveTime, TimeUnit.SECONDS,
> new NamedThreadFactory("XXX"+name));
>
> ChannelPipeline pipeline = bootstrap.getPipeline();
> pipeline.addLast("exec", new ExecutionHandler(threadPool));
> pipeline.addLast("decoder", new AsyncMSGIoDecoder());
> pipeline.addLast("encoder", new AsyncMSGIoEncoder());
> pipeline.addLast("handler", new MSGIoHandler(this));
>
> channelFuture = bootstrap.connect(new
> InetSocketAddress(host, port));
>
>
> Thanks in advance,
> JW
> _______________________________________________
> netty-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/netty-users
>

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

— Trustin Lee, http://gleamynode.net/

synchronous client send/rcv help

2009-11-18T14:24:44Z

I'm looking for some advice on a good design for the following problem.

I have a need to perform an app-level handshake once the channel is
connected. I've implemented this via a handshake method call fired from
the channelConnected(ChannelHandlerContext ctx, ChannelStateEvent e) of
my IoHandler (extending SimpleChannelUpstreamHandler).

The crux of the issue is a need for this app-level handshake to occur
synchronously blocking each subsequent send (in the handshake) until a
previous response from the peer is rcvd and validated. I've found that
blocking during my handshake method in fact blocks the I/O thread
altogether. I can see the peer actually receives the req and sends a
response, but a corresponding messageReceived(ChannelHandlerContext ctx,
MessageEvent e) isn't fired until after my originating caller times
out... which just so happens to initiate a disconnect and disposal of
resources as well.

I can easily workaround this issue by chaining messageReceived
callbacks, but I'm looking for a more elegant solution. Does anyone have
advice on how I can pull this off? Is there any example of how to force
only some requests to be blocking? I've seen the thread on the forum
where Trustin indicates adding a LinkedBlockingQueue is a solution, but
the original poster's use-case is quite different from mine in that all
requests needed to be blocking, and as best I understand it, I'm
effectively doing the same thing but creating a problem because I'm
calling my blocking method from channelConnected()... altho I might be
totally wrong.

Any advice is greatly appreciated.

Note : Because this might be a simple problem with my I/O thread pools,
I'll include my client code here (only slightly redacted for protected
info).

ClientSocketChannelFactory factory = new
NioClientSocketChannelFactory(
Executors.newCachedThreadPool(new
NamedThreadFactory("NioClientSocketChannelFactoryBoss_"+name)),
Executors.newCachedThreadPool(new
NamedThreadFactory("NioClientSocketChannelFactory_"+name))
);
bootstrap = new ClientBootstrap(factory);
bootstrap.setOption("tcpNoDelay", true);
bootstrap.setOption("keepAlive", true);

OrderedMemoryAwareThreadPoolExecutor threadPool = new
OrderedMemoryAwareThreadPoolExecutor(
corePoolSize, 0, 0, keepAliveTime, TimeUnit.SECONDS,
new NamedThreadFactory("XXX"+name));

ChannelPipeline pipeline = bootstrap.getPipeline();
pipeline.addLast("exec", new ExecutionHandler(threadPool));
pipeline.addLast("decoder", new AsyncMSGIoDecoder());
pipeline.addLast("encoder", new AsyncMSGIoEncoder());
pipeline.addLast("handler", new MSGIoHandler(this));

channelFuture = bootstrap.connect(new
InetSocketAddress(host, port));

Thanks in advance,
JW
_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

Re: Postponing of ChunkedWriteHandlers hasNextChunk retrieval after write operation

2009-11-21T00:12:47Z

Hi Tuomas,

On Wed, Nov 18, 2009 at 9:01 PM, Tuomas Kiviaho <[hidden email]> wrote:
>
> I was having a deadlock in my simple unit test using PipedInputStream and
> PipedOutputStream with ChunkedWriteHandler. I discovered that
> ChunkedWriteHandler checks for next chunk prior writing out the current
> chunk.

Are you mentioning the 182nd line of ChunkedWriteHandler that calls
ChunkedInput.hasNextChunk()? It would be nice if you could include
the full thread dump.

> What if next chunk check is postponed after write operation? It doesn't seem
> to be impossible at first glance but I guess there's a good reason for this
> kind of implementation.

Yes, the next chunk check could be postponed after Channels.write(...)
call at line 214. However, before making the modification, I'd like
to know why dead lock occurs.

Thanks
— Trustin Lee, http://gleamynode.net/

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

— Trustin Lee, http://gleamynode.net/

Postponing of ChunkedWriteHandlers hasNextChunk retrieval after write operation

2009-11-18T04:01:33Z

I was having a deadlock in my simple unit test sing PipedInputStream and PipedOutputStream with ChunkedWriteHandler. I discovered that ChunkedWriteHandler checks for next chunk prior writing out the current chunk.

What if next chunk check is postponed after write operation? It doesn't seem to be impossible at first glance but I guess there's a good reason for this kind of implementation.

--
Tuomas

Re: http tunnel not as servlet

2009-11-21T02:45:26Z

Hi Adam,

I'll def take a look at your LittleProxy. Still I'd appreciate some help
with netty's native support for tunnelling and having this feature build-in
into server apart from having it as deployed as servlet.

Best,

Pete

----- Original Message -----
From: "Adam Fisk" <[hidden email]>
To: "Netty Users" <[hidden email]>
Sent: Wednesday, November 18, 2009 9:13 AM
Subject: Re: http tunnel not as servlet

> LittleProxy supports tunneling. It's still in its infancy, but it's
> been working well in our tests:
>
> http://dev.littleshoot.org:8084/display/proxy/LittleShoot+Java+HTTP+Proxy+-+LittleProxy
>
> -Adam
>
>
> On Tue, Nov 17, 2009 at 12:24 PM, miarkus <[hidden email]> wrote:
>>
>> Please advise me what steps should I take to have http tunnel built-in
>> into
>> NETTY server, so it won't have to run as servlet.
>>
>> Thanks
>> --
>> View this message in context:
>> http://n2.nabble.com/http-tunnel-not-as-servlet-tp4021041p4021041.html
>> Sent from the Netty User Group mailing list archive at Nabble.com.
>> _______________________________________________
>> netty-users mailing list
>> [hidden email]
>> https://lists.jboss.org/mailman/listinfo/netty-users
>>
>
>
>
> --
> Adam Fisk
> http://www.littleshoot.org | http://adamfisk.wordpress.com |
> http://twitter.com/adamfisk
> _______________________________________________
> netty-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/netty-users

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

Re: http tunnel not as servlet

2009-11-18T00:13:45Z

LittleProxy supports tunneling. It's still in its infancy, but it's
been working well in our tests:

http://dev.littleshoot.org:8084/display/proxy/LittleShoot+Java+HTTP+Proxy+-+LittleProxy

-Adam

On Tue, Nov 17, 2009 at 12:24 PM, miarkus <[hidden email]> wrote:

>
> Please advise me what steps should I take to have http tunnel built-in into
> NETTY server, so it won't have to run as servlet.
>
> Thanks
> --
> View this message in context: http://n2.nabble.com/http-tunnel-not-as-servlet-tp4021041p4021041.html
> Sent from the Netty User Group mailing list archive at Nabble.com.
> _______________________________________________
> netty-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/netty-users
>

--
Adam Fisk
http://www.littleshoot.org | http://adamfisk.wordpress.com |
http://twitter.com/adamfisk
_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

http tunnel not as servlet

2009-11-17T12:24:14Z

Please advise me what steps should I take to have http tunnel built-in into NETTY server, so it won't have to run as servlet.

Thanks

Re: Netty 3.2.0.ALPHA1 released: New Journey to the Next Level

2009-11-18T00:15:58Z

Great work Trustin. The compression and decompression handlers are
quite a feat - looking forward to checking out the code!

-Adam

On Mon, Nov 16, 2009 at 5:39 AM, Trustin Lee (이희승) <[hidden email]> wrote:

> Hi folks,
>
> Netty 3.2.0.ALPHA1, the first milestone in the 3.2 branch, has been released.
>
> The Netty project is an effort to provide an asynchronous,
> event-driven network application framework and tools for rapid
> development of maintainable, high-performance, high-scalability
> protocol servers and clients.
>
> In other words, Netty is a NIO client server framework that enables
> quick and easy development of network applications such as protocol
> servers and clients. It simplifies and streamlines network programming
> such as TCP and UDP socket servers
>
> Please visit our web site to download it. Your feed back is more than
> appreciated - please visit the discussion forum and leave some
> comments and questions:
>
> * http://www.jboss.org/netty/
>
> Compression Handlers
> ====================
> The ability to compress and decompress a connection transparently was
> one of the early feature requests in Netty development. The
> compression handlers introduced in this release add complete support
> for both GZIP and ZLIB compression encoding, which is not found in any
> other NIO application frameworks. For more information, please look
> at the org.jboss.netty.handler.codec.compression package.
>
> On top of the new compression handlers, HTTP content compression /
> decompression support has been added, too. Please take a look at the
> HTTP snoop client / server example for more information.
>
> New Testimonial
> ===============
> Dave Siracusa from Perfware LLC sent a nice testimonial for Netty
> recently. You will find it very interesting; he succeeded to write an
> HTTP server which is capable of processing 15,000 messages per second
> on top of Netty. Check his recommendation and other testimonials
> here: http://www.jboss.org/netty/testimonials.html
>
> Upcoming Releases
> =================
> Although we did not add many new features in this first alpha, we have
> bunch of new features planned, and they are all public in our issue
> tracker and mailing list. Please feel free to join the discussion and
> keep us updated with any ideas for new features, bugs, design issues,
> API usability feed back, and questions - your idea is the future of
> Netty!
>
> Miscellaneous
> =============
> For the full changelog, please visit here:
>
> * http://jira.jboss.org/jira/browse/NETTY
>
> Cheers,
> Trustin
>
> — Trustin Lee, http://gleamynode.net/
>
> _______________________________________________
> netty-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/netty-users
>

--
Adam Fisk
http://www.littleshoot.org | http://adamfisk.wordpress.com |
http://twitter.com/adamfisk

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

Netty 3.2.0.ALPHA1 released: New Journey to the Next Level

2009-11-16T05:39:55Z

Hi folks,

Netty 3.2.0.ALPHA1, the first milestone in the 3.2 branch, has been released.

The Netty project is an effort to provide an asynchronous,
event-driven network application framework and tools for rapid
development of maintainable, high-performance, high-scalability
protocol servers and clients.

In other words, Netty is a NIO client server framework that enables
quick and easy development of network applications such as protocol
servers and clients. It simplifies and streamlines network programming
such as TCP and UDP socket servers

Please visit our web site to download it. Your feed back is more than
appreciated - please visit the discussion forum and leave some
comments and questions:

* http://www.jboss.org/netty/

Compression Handlers
====================
The ability to compress and decompress a connection transparently was
one of the early feature requests in Netty development. The
compression handlers introduced in this release add complete support
for both GZIP and ZLIB compression encoding, which is not found in any
other NIO application frameworks. For more information, please look
at the org.jboss.netty.handler.codec.compression package.

On top of the new compression handlers, HTTP content compression /
decompression support has been added, too. Please take a look at the
HTTP snoop client / server example for more information.

New Testimonial
===============
Dave Siracusa from Perfware LLC sent a nice testimonial for Netty
recently. You will find it very interesting; he succeeded to write an
HTTP server which is capable of processing 15,000 messages per second
on top of Netty. Check his recommendation and other testimonials
here: http://www.jboss.org/netty/testimonials.html

Upcoming Releases
=================
Although we did not add many new features in this first alpha, we have
bunch of new features planned, and they are all public in our issue
tracker and mailing list. Please feel free to join the discussion and
keep us updated with any ideas for new features, bugs, design issues,
API usability feed back, and questions - your idea is the future of
Netty!

Miscellaneous
=============
For the full changelog, please visit here:

* http://jira.jboss.org/jira/browse/NETTY

Cheers,
Trustin

— Trustin Lee, http://gleamynode.net/

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

— Trustin Lee, http://gleamynode.net/

Re: Netty client in Java 1.4

2009-11-15T06:06:36Z

Virat,
Thanks for the reply. Of course... Makes perfect sense.
I was blinded by the cool technology. But, I won't be able
to pad my resume ;(.

Aaron

Virat Gohil-3 wrote:

Hi,

There is a third option too:

3. Write the application using the Old I/O (blocking) approach.

Netty can work with non-NIO clients as well. Write your application as
if you are communicating to a regular server.

HTH,

Virat

On Sat, Nov 14, 2009 at 11:01 PM, tstat <asmith@proliphix.com> wrote:
>
> I have been tasked to write a client that will be talking to a server running
> Netty.
> Unfortunately, the client environment only provides Java 1.4.2.
> I have attempted to use Retrotranslator but have met with many
> exceptions and unexplained crashes. Does anyone know of what might
> be an alternative.
>
> 1. JBoss Retro?
> 2. Bite the bullet and write the client using regular NIO etc...
>
> Thanks for your help.
> --
> View this message in context: http://n2.nabble.com/Netty-client-in-Java-1-4-tp4004933p4004933.html
> Sent from the Netty User Group mailing list archive at Nabble.com.
> _______________________________________________
> netty-users mailing list
> netty-users@lists.jboss.org
> https://lists.jboss.org/mailman/listinfo/netty-users
>

_______________________________________________
netty-users mailing list
netty-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/netty-users

Re: Netty client in Java 1.4

2009-11-14T21:07:28Z

Hi,

There is a third option too:

3. Write the application using the Old I/O (blocking) approach.

Netty can work with non-NIO clients as well. Write your application as
if you are communicating to a regular server.

HTH,

Virat

On Sat, Nov 14, 2009 at 11:01 PM, tstat <[hidden email]> wrote:

>
> I have been tasked to write a client that will be talking to a server running
> Netty.
> Unfortunately, the client environment only provides Java 1.4.2.
> I have attempted to use Retrotranslator but have met with many
> exceptions and unexplained crashes. Does anyone know of what might
> be an alternative.
>
> 1. JBoss Retro?
> 2. Bite the bullet and write the client using regular NIO etc...
>
> Thanks for your help.
> --
> View this message in context: http://n2.nabble.com/Netty-client-in-Java-1-4-tp4004933p4004933.html
> Sent from the Netty User Group mailing list archive at Nabble.com.
> _______________________________________________
> netty-users mailing list
> [hidden email]
> https://lists.jboss.org/mailman/listinfo/netty-users
>

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

Netty client in Java 1.4

2009-11-14T09:31:13Z

I have been tasked to write a client that will be talking to a server running Netty.
Unfortunately, the client environment only provides Java 1.4.2.
I have attempted to use Retrotranslator but have met with many
exceptions and unexplained crashes. Does anyone know of what might
be an alternative.

1. JBoss Retro?
2. Bite the bullet and write the client using regular NIO etc...

Thanks for your help.

Re: GPS trackers

2009-11-17T04:03:22Z

Hi QP,

On Sat, Nov 14, 2009 at 8:40 AM, qp <[hidden email]> wrote:
>
> Hi,
>
> I have a dude. There are many GPS devices that send locations. Is possible
> to configure ip and port but can't install a some socket client. So probably
> they sent locations in standard binary sockets. I don't now much about nio,
> but I suppose that to use nio you have to use client that support nio.

As long as the GPS devices communicate via TCP/IP or UDP/IP, you can
use any networking API (NIO, old java.io, C, whatever) to communicate
with them. You don't need to use NIO on the both side.

> Using netty, can I process the incoming sockets from the GPS devices?

Sure you can, if the GPS device speak TCP/IP or UDP/IP.

HTH,

— Trustin Lee, http://gleamynode.net/

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

— Trustin Lee, http://gleamynode.net/

GPS trackers

2009-11-13T15:40:16Z

Hi,

I have a dude. There are many GPS devices that send locations. Is possible to configure ip and port but can't install a some socket client. So probably they sent locations in standard binary sockets. I don't now much about nio, but I suppose that to use nio you have to use client that support nio.

Using netty, can I process the incoming sockets from the GPS devices?

Re: Usefull ByteBuffer methods are not present in ChannelBuffer

2009-11-17T00:13:56Z

On Tue, Nov 17, 2009 at 4:26 PM, Atra <[hidden email]> wrote:

>
>
> Thomas Bocek wrote:
>>
>> Hi Atra,
>>
>> You can use longBitsToDouble(long), doubleToRawLongBits(double value),
>> doubleToLongBits(double value), etc. and use putLong()/getLong()
>>
>> Cheers,
>>
>> Thomas
>>
>>
>
> Hi Thomas,
>
> thanks for the usefull hint. Yes, with float and double this will work. With
> char it is also possible to do some bits operations (just looking at
> ByteBuffer source). But the general idea is to not reinvent the wheel. I'm
> pretty sure everyone who deals with some custom protocol needs those methods
> and its not good to reinvent the wheel.
>
> Btw may be I should create a feature request in JIRA?

That would be great. Thanks in advance! :)

Cheers

— Trustin Lee, http://gleamynode.net/

_______________________________________________
netty-users mailing list
[hidden email]
https://lists.jboss.org/mailman/listinfo/netty-users

— Trustin Lee, http://gleamynode.net/

Re: Usefull ByteBuffer methods are not present in ChannelBuffer

2009-11-16T23:26:47Z

Thomas Bocek wrote:

Hi Atra,

You can use longBitsToDouble(long), doubleToRawLongBits(double value),
doubleToLongBits(double value), etc. and use putLong()/getLong()

Cheers,

Thomas

Hi Thomas,

thanks for the usefull hint. Yes, with float and double this will work. With char it is also possible to do some bits operations (just looking at ByteBuffer source). But the general idea is to not reinvent the wheel. I'm pretty sure everyone who deals with some custom protocol needs those methods and its not good to reinvent the wheel.

Btw may be I should create a feature request in JIRA?

Regards