OSGeo.org › OSGeo Software and Data Projects › deegree › deegree - devel

Re: Status on all running Processes

13 messages

Re: Status on all running Processes

Reply Threaded

Christian,

> If we think in more detail about it, this list
> may grow and we put a year of work into it :-) Anyway, your requirement
> is a good starting point for this task and we can add more functionality
> later.

This will be good to keep in mind as we're working, but I think I'll start with the simple stuff first. :)

> If stuff gets too complicated to discuss by mail, we can also offer a
> telco during the next week. Just let us know.

This is good to know. You guys are in Germany, right? I'm out in Colorado, USA -- about a 6 hour difference.

Marcus,

Thanks for the adding the wiki pages.

> I would rather go
> for simple JSP pages that run in the same webapp as the service (and
> could become a standard-part of deegree 3 service installations in the
> end). For the communication with the services, I would suggest using
> plain Java method calls.

I like JSP page approach; it certainly simplifies things. On that same note, requiring that the monitoring app be part of the same webapp seems natural to me. Especially since it is so easy to extend and include Deegree in a custom webapp.

> For just providing the information on running processes, we don't
> necessarily need a database though (although this may be the most
> professional solution in the end). The WPSController/WPService could
> just track this information and provide them via get-methods (maybe this
> should be delegated to a separate class, e.g. ProcessManager). If you
> query this from a JSP (as mentioned above), you're almost there...

I'm a bit hesitant about storing process information in memory only vs. persistent storage. I'm afraid I may not have been clear in my original post. It'd be nice to get a bit more than just the list of running processes. I see the following information as useful:

Retrieve the status on [all, running, completed] processes

state (pending, running)
Progress
Status Location URL - for additional info

For my application several of my users (including myself) like to go back and look at the history of processes. E.g. What is the provenance of this WMS link? Obviously, if implemented, this would have to be a configurable parameter in case a WPS instance manages very large numbers of hits -- most people wouldn't want to store the history of a million processes.

Of course the main detractor to persistent storage that I'm aware of, albeit modest, is complexity.

Cheers,

-jason

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
deegree-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/deegree-devel

Markus Schneider-4

Re: Status on all running Processes

Reply Threaded

Hi Jason,

Jason R. Surratt wrote:

> I'm a bit hesitant about storing process information in memory only vs.
> persistent storage. I'm afraid I may not have been clear in my original
> post. It'd be nice to get a bit more than just the list of running
> processes. I see the following information as useful:
>
> * Retrieve the status on [all, running, completed] processes
> o state (pending, running)
> o Progress
> o Status Location URL - for additional info
>

I agree, this all sounds useful. Can you add this to the corresponding section of the wiki page?

> For my application several of my users (including myself) like to go
> back and look at the history of processes. E.g. What is the provenance
> of this WMS link? Obviously, if implemented, this would have to be a
> configurable parameter in case a WPS instance manages very large numbers
> of hits -- most people wouldn't want to store the history of a million
> processes.
>
> Of course the main detractor to persistent storage that I'm aware of,
> albeit modest, is complexity.

Maybe we should just start with implementing the in-memory approach first and add persistent storage for the process
execution information in the next step. IMHO, the two aspects

1. Retrieving information via appropriate methods from a ProcessManager + display in JSP
2. Use persistent storage for ProcessManager

are quite isolated, so this appears to be the natural order to me.

Best regards,
Markus

--
Markus Schneider

l a t / l o n GmbH
Aennchenstrasse 19 53177 Bonn, Germany
phone ++49 +228 184960 fax ++49 +228 1849629
http://www.lat-lon.de http://www.deegree.org

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
deegree-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/deegree-devel

signature.asc (268 bytes) Download Attachment

Jason R. Surratt

Re: Status on all running Processes

Reply Threaded

I agree, this all sounds useful. Can you add this to the corresponding section of the wiki page?

Will do.

Maybe we should just start with implementing the in-memory approach first and add persistent storage for the process
execution information in the next step. IMHO, the two aspects

1. Retrieving information via appropriate methods from a ProcessManager + display in JSP
2. Use persistent storage for ProcessManager

Sounds good to me. I'll write up my intentions on the wiki and if there aren't any objections I'll get started on a first cut.

Thanks!

-jason

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
deegree-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/deegree-devel

Jason R. Surratt

Re: Status on all running Processes

Reply Threaded

I agree, this all sounds useful. Can you add this to the corresponding section of the wiki page?

Will do.

Maybe we should just start with implementing the in-memory approach first and add persistent storage for the process
execution information in the next step. IMHO, the two aspects

1. Retrieving information via appropriate methods from a ProcessManager + display in JSP
2. Use persistent storage for ProcessManager

Sounds good to me. I'll write up my intentions on the wiki and if there aren't any objections I'll get started on a first cut.

I have updated the wiki:

https://wiki.deegree.org/deegreeWiki/deegree3/ServiceManagementConsolePlans
https://wiki.deegree.org/deegreeWiki/deegree3/WebProcessingServiceInternals

Please let me know what you think.

Thanks!

-jason

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
deegree-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/deegree-devel

Markus Schneider-4

Re: Status on all running Processes

Reply Threaded

Hi Jason,

Jason R. Surratt wrote:

> I have updated the wiki:
>
> https://wiki.deegree.org/deegreeWiki/deegree3/ServiceManagementConsolePlans

Looks good to me :-) Two little remarks:

- In 3.1, I guess you mean ProcessletExecution, not ProcessletExecutionInfo? The ProcessletExecutionInfo is basically
only a part of the ProcessletExecution that the Processlet can access -- and the monitor would probably need to access
all available information.
- I just realized that we maybe we don't need a new ProcessManager class after all, as the ExecutionManager is partly
doing it's job already and could be extended easily to keep a list of all ProcessletExecution instances (also the
finished ones) . Of course, if this gets to messy, we can still split up the logic into different classes.

Best regards,
Markus

> https://wiki.deegree.org/deegreeWiki/deegree3/WebProcessingServiceInternals
>
> Please let me know what you think.
>
> Thanks!
>
> -jason
>
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now. http://p.sf.net/sfu/bobj-july
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> deegree-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/deegree-devel

--
Markus Schneider

l a t / l o n GmbH
Aennchenstrasse 19 53177 Bonn, Germany
phone ++49 +228 184960 fax ++49 +228 1849629
http://www.lat-lon.de http://www.deegree.org

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
deegree-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/deegree-devel

signature.asc (268 bytes) Download Attachment

Jason R. Surratt

Re: Status on all running Processes

Reply Threaded

> I have updated the wiki:
>
> https://wiki.deegree.org/deegreeWiki/deegree3/ServiceManagementConsolePlans

Looks good to me :-) Two little remarks:

- In 3.1, I guess you mean ProcessletExecution, not ProcessletExecutionInfo? The ProcessletExecutionInfo is basically
only a part of the ProcessletExecution that the Processlet can access -- and the monitor would probably need to access
all available information.
- I just realized that we maybe we don't need a new ProcessManager class after all, as the ExecutionManager is partly
doing it's job already and could be extended easily to keep a list of all ProcessletExecution instances (also the
finished ones) . Of course, if this gets to messy, we can still split up the logic into different classes.

Marcus,

Good call. I've updated the wiki with those changes.

Should I just post a patch to the mailing list when I've got a first cut of my changes?

Thanks!

-jason

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
deegree-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/deegree-devel

Markus Schneider-4

Re: Status on all running Processes

Reply Threaded

Jason,

> Good call. I've updated the wiki with those changes.
>
> Should I just post a patch to the mailing list when I've got a first cut
> of my changes?

Yes, just send it to list. When this gets too cumbersome, we can set up an SVN commit account for you.

Best regards,
Markus

--
Markus Schneider

l a t / l o n GmbH
Aennchenstrasse 19 53177 Bonn, Germany
phone ++49 +228 184960 fax ++49 +228 1849629
http://www.lat-lon.de http://www.deegree.org

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
deegree-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/deegree-devel

signature.asc (268 bytes) Download Attachment

Jason R. Surratt

Re: Status on all running Processes

Reply Threaded

> Good call. I've updated the wiki with those changes.
>
> Should I just post a patch to the mailing list when I've got a first cut
> of my changes?

Yes, just send it to list. When this gets too cumbersome, we can set up an SVN commit account for you.

Marcus,

The patch is attached. I've highlighted some things of note below.

I didn't see any straight forward way to get an instance of OGCFrontController from the .jsp files. I've added a quick hack in for now that should definitely be removed. It appears that ServletContext#getServlet used to provide this functionality but it has since been deprecated. My suggestion is to refactor OGCFrontController into two classes, a singleton that provides the bulk of the functionality and a servlet. However, I didn't feel comfortable hacking and slashing Deegree that much w/o mentioning it first.
It appears that the "responseDocumentIdToState" HashMap could have suffered from concurrency issues within Processworker#run, I've converted it to a ConcurrentHashMap to avoid the problem.
I am ashamed to say I didn't add any unit tests in yet, but I'll get that done tomorrow. Due to the time difference I wanted to make sure I got something off to you tonight.
I've included two .jsp files, one that lists the processes in as very simple html and another that response with simple XML (the functionality I need)

Please let me know what you think. Thanks!

-jason

Index: resources/client-template/client/processList.jsp
===================================================================
--- resources/client-template/client/processList.jsp (revision 0)
+++ resources/client-template/client/processList.jsp (revision 0)
@@ -0,0 +1,134 @@
+<%-- $HeadURL: svn+ssh://[hidden email]/deegree/apps/services-template/trunk/web/client/client.jsp $ --%>
+<%-- $Id: client.jsp 12467 2008-06-20 16:52:29Z jmays $ --%>
+<%@ page language="java" contentType="text/html; charset=UTF-8" pageEncoding="UTF-8" %>
+<%@ page import="org.deegree.commons.version.*"%>
+<%@ page import="org.deegree.services.controller.OGCFrontController"%>
+<%@ page import="org.deegree.services.controller.wps.WPSController"%>
+<%@ page import="org.deegree.services.controller.wps.ProcessletExecution"%>
+<%@ page import="java.io.File"%>
+<%@ page import="java.io.FilenameFilter"%>
+<%@ page import="java.util.Arrays"%>
+<%@ page import="java.util.Collection"%>
+<%@ page import="java.util.Iterator"%>
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
+<html>
+ <head>
+ <meta http-equiv="content-type" content="text/html; charset=UTF-8">
+ <title>WPS Process List Client</title>
+ <link rel="stylesheet" href="deegree.css" />
+ </head>
+ <body leftmargin="10" rightmargin="10" topmargin="0" marginwidth="0" marginheight="0"
+ link="#2D2A63" vlink="#2D2A63" alink="#E31952">
+ <table width="100%" border="0" cellpadding="0" cellspacing="0">
+ <tbody>
+ <tr valign="top" bgColor="#fe5215">
+ <td align="left" noWrap>
+ <font size="2"> <b>deegree Version: </b> </font>
+ 
+ </td>
+ <td align="right" noWrap>
+ <font size="2">
+ <b>more info at <a href="http://www.deegree.org/">http://www.deegree.org</a>.
+ Copyright by <a href="http://www.lat-lon.de">lat/lon GmbH</a>
+ and <a href="http://aggis.uni-bonn.de">Bonn University</a> </b>
+ </font>
+ </td>
+ </tr>
+ <tr>
+ <td align="center" colspan="2">
+ <br /><h2>deegr<font color="#fe5215">ee</font> OGC WPS Process List Client</h2>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+ <center>
+ <table width="70%" border="1" cellpadding="3" cellspacing="0">
+ <tbody>
+ <tr>
+ <td>Identifier</td>
+ <td>Status</td>
+ <td>Progress</td>
+ <td>Start Time</td>
+ <td>Finish Time</td>
+ <td>Duration</td>
+ <td>Message</td>
+ </tr>
+<%
+ OGCFrontController frontController = OGCFrontController.me;
+ WPSController controller = (WPSController)frontController.getServiceController(WPSController.class);
+
+ Collection<ProcessletExecution> allProcesses = controller.getAllProcessStates();
+
+ java.text.SimpleDateFormat df = new java.text.SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
+
+
+ for (ProcessletExecution p : allProcesses)
+ {
+ String durationStr = "";
+ long duration = -1;
+ if (p.getFinishTime() > 0)
+ {
+ duration = p.getFinishTime() - p.getStartTime();
+ }
+ else if (p.getStartTime() > 0)
+ {
+ duration = new java.util.Date().getTime() - p.getStartTime();
+ }
+ if (duration >= 0)
+ {
+ duration /= 1000;
+ long seconds = duration % 60;
+ long minutes = (duration % 3600) / 60;
+ long hours = duration / 3600;
+ durationStr = String.format("%02d:%02d:%02d", hours, minutes, seconds);
+ }
+
+ String message = "";
+ if (p.getAcceptedMessage() != null)
+ {
+ message += "<p>Accepted: " + p.getAcceptedMessage() + "</p>\n";
+ }
+ if (p.getStartMessage() != null)
+ {
+ message += "<p>Start: " + p.getStartMessage() + "</p>\n";
+ }
+ if (p.getPauseMessage() != null)
+ {
+ message += "<p>Pause: " + p.getPauseMessage() + "</p>\n";
+ }
+ if (p.getSucceededMessage() != null)
+ {
+ message += "<p>Succeeded: " + p.getSucceededMessage() + "</p>\n";
+ }
+ if (p.getFailedException() != null)
+ {
+ message += "<p>Failed: " + p.getFailedException().getMessage() + "</p>\n";
+ }
+
+ String statusUrl = "";
+ if (p.getResponseStorageUrl() != null)
+ {
+ statusUrl = "<a href='" + p.getResponseStorageUrl().toString() + "'>Status</a>";
+ }
+
+ %>
+ <tr>
+ <td><%= p.getProcessId().toString() %></td>
+ <td><%= p.getExecutionState().toString() %></td>
+ <td><%= p.getPercentCompleted() %></td>
+ <td><%= df.format(p.getStartTime()) %></td>
+ <td><%= p.getFinishTime() > 0 ? df.format(p.getFinishTime()) : "" %></td>
+ <td><%= durationStr %></td>
+ <td><%= message %></td>
+ <td><%= statusUrl %></td>
+ </tr>
+ <%
+ }
+%>
+ </tbody>
+ </table>
+ </center>
+ </body>
+</html>
Index: resources/client-template/client/processListXml.jsp
===================================================================
--- resources/client-template/client/processListXml.jsp (revision 0)
+++ resources/client-template/client/processListXml.jsp (revision 0)
@@ -0,0 +1,80 @@
+<%@ page language="java" contentType="text/xml; charset=UTF-8" pageEncoding="UTF-8" %>
+<%@ page import="org.deegree.commons.version.*"%>
+<%@ page import="org.deegree.services.controller.OGCFrontController"%>
+<%@ page import="org.deegree.services.controller.wps.WPSController"%>
+<%@ page import="org.deegree.services.controller.wps.ProcessletExecution"%>
+<%@ page import="java.io.File"%>
+<%@ page import="java.io.FilenameFilter"%>
+<%@ page import="java.util.Arrays"%>
+<%@ page import="java.util.Collection"%>
+<%@ page import="java.util.Iterator"%>
+<AllProcesses>
+<%
+ OGCFrontController frontController = OGCFrontController.me;
+ WPSController controller = (WPSController)frontController.getServiceController(WPSController.class);
+
+ Collection<ProcessletExecution> allProcesses = controller.getAllProcessStates();
+
+ java.text.SimpleDateFormat df = new java.text.SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ");
+
+
+ for (ProcessletExecution p : allProcesses)
+ {
+ String durationStr = "";
+ long duration = -1;
+ if (p.getFinishTime() > 0)
+ {
+ duration = p.getFinishTime() - p.getStartTime();
+ }
+ else if (p.getStartTime() > 0)
+ {
+ duration = new java.util.Date().getTime() - p.getStartTime();
+ }
+ if (duration >= 0)
+ {
+ durationStr = String.format("%d", duration);
+ }
+
+ String message = "";
+ if (p.getAcceptedMessage() != null)
+ {
+ message += "<p>Accepted: " + p.getAcceptedMessage() + "</p>\n";
+ }
+ if (p.getStartMessage() != null)
+ {
+ message += "<p>Start: " + p.getStartMessage() + "</p>\n";
+ }
+ if (p.getPauseMessage() != null)
+ {
+ message += "<p>Pause: " + p.getPauseMessage() + "</p>\n";
+ }
+ if (p.getSucceededMessage() != null)
+ {
+ message += "<p>Succeeded: " + p.getSucceededMessage() + "</p>\n";
+ }
+ if (p.getFailedException() != null)
+ {
+ message += "<p>Failed: " + p.getFailedException().getMessage() + "</p>\n";
+ }
+
+ String statusUrl = "";
+ if (p.getResponseStorageUrl() != null)
+ {
+ statusUrl = "<a href='" + p.getResponseStorageUrl().toString() + "'>Status</a>";
+ }
+
+ %>
+ <Process>
+ <Identifier><%= p.getProcessId().toString() %></Identifier>
+ <Status><%= p.getExecutionState().toString() %></Status>
+ <Progress><%= p.getPercentCompleted() %></Progress>
+ <StartTime><%= df.format(p.getStartTime()) %></StartTime>
+ <FinishTime><%= p.getFinishTime() > 0 ? df.format(p.getFinishTime()) : "" %></FinishTime>
+ <Duration><%= durationStr %></Duration>
+ <SucceededMessage><%= p.getSucceededMessage() == null ? "" : p.getSucceededMessage() %></SucceededMessage>
+ <FailedMessage><%= p.getFailedException() == null ? "" : p.getFailedException().getMessage() %></FailedMessage>
+ </Process>
+ <%
+ }
+%>
+</AllProcesses>
\ No newline at end of file
Index: src-api/org/deegree/services/controller/AbstractOGCServiceController.java
===================================================================
--- src-api/org/deegree/services/controller/AbstractOGCServiceController.java (revision 19222)
+++ src-api/org/deegree/services/controller/AbstractOGCServiceController.java (working copy)
@@ -86,7 +86,7 @@
import org.xml.sax.SAXException;

/**
- * Each conrete extension of this class is responsible for handling requests to a specific OGC web service (WPS, WMS,
+ * Each concrete extension of this class is responsible for handling requests to a specific OGC web service (WPS, WMS,
* WFS, CSW,...).
*
* @see OGCFrontController
Index: src-api/org/deegree/services/controller/OGCFrontController.java
===================================================================
--- src-api/org/deegree/services/controller/OGCFrontController.java (revision 19222)
+++ src-api/org/deegree/services/controller/OGCFrontController.java (working copy)
@@ -172,6 +172,9 @@

private static final InheritableThreadLocal<RequestContext> CONTEXT = new InheritableThreadLocal<RequestContext>();

+ // TODO Remove me.
+ public static OGCFrontController me;
+
/**
* Returns the {@link RequestContext} associated with the calling thread.
* <p>
@@ -753,11 +756,33 @@
private AbstractOGCServiceController determineResponsibleControllerByNS( String ns ) {
return serviceNSToController.get( ns );
}
+
+ /**
+ * Return a service controller instance based on the class of the service.
+ * @param c
+ * Class of the service requested. E.g. WPSController.getClass()
+ * @return Returns the instance of the requested service used by OGCFrontController, or null if the service is
+ * not registered.
+ */
+ public AbstractOGCServiceController getServiceController(Class<? extends AbstractOGCServiceController> c)
+ {
+ AbstractOGCServiceController result = null;
+ for (AbstractOGCServiceController it : serviceNSToController.values())
+ {
+ if (c == it.getClass())
+ {
+ result = it;
+ break;
+ }
+ }
+ return result;
+ }

@Override
public void init( ServletConfig config )
throws ServletException {
-
+ // TODO Remove me.
+ me = this;
super.init( config );
String configURL = getInitParameter( "ServicesConfiguration" );
if ( configURL == null ) {
Index: src/org/deegree/services/controller/wps/ExecutionManager.java
===================================================================
--- src/org/deegree/services/controller/wps/ExecutionManager.java (revision 19222)
+++ src/org/deegree/services/controller/wps/ExecutionManager.java (working copy)
@@ -40,11 +40,13 @@
import java.io.OutputStreamWriter;
import java.net.URL;
import java.util.ArrayList;
+import java.util.Collection;
import java.util.Collections;
import java.util.Date;
-import java.util.HashMap;
import java.util.List;
import java.util.Map;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

@@ -96,14 +98,18 @@

private static final Logger LOG = LoggerFactory.getLogger( ExecutionManager.class );

+ // Is this a legacy variable? It doesn't appear to be used.
private WPSController master;

private StorageManager storageManager;

private ExecutorService exec;
+
+ // A list of all processes that have been run or are currently running.
+ private Collection<ProcessletExecution> processStateList = new ConcurrentLinkedQueue<ProcessletExecution>();

// key: response document of the process (currently running), value: status object
- private Map<ResponseDocumentStorage, ProcessletExecution> responseDocumentIdToState = new HashMap<ResponseDocumentStorage, ProcessletExecution>();
+ private Map<ResponseDocumentStorage, ProcessletExecution> responseDocumentIdToState = new ConcurrentHashMap<ResponseDocumentStorage, ProcessletExecution>();

/**
* Creates a new {@link ExecutionManager} for a {@link WPSController}.
@@ -119,6 +125,40 @@
this.storageManager = storageManager;
this.exec = Executors.newCachedThreadPool();
}
+
+ /**
+ * This method should be called in lieu of the constructor directly to ensure the ProcessExecution gets added
+ * to the processList. See {@link ProcessletExecution} for a definition of the parameters.
+ */
+ private ProcessletExecution createProcessletExecution(ExecuteRequest request,
+ StorageLocation responseStorage, URL serviceInstance, List<RequestedOutput> outputParams,
+ ProcessletOutputs outputs)
+ {
+ ProcessletExecution result = new ProcessletExecution(request, responseStorage, serviceInstance, outputParams,
+ outputs);
+ processStateList.add(result);
+ return result;
+ }
+
+ /**
+ * Returns a collection of information on all processes, including processes that haven't run, are running and
+ * have already stopped.
+ * @return Returns a collection of information on all processes, including processes that haven't run, are running
+ * and have already stopped. The returned result should not be modified.
+ */
+ Collection<ProcessletExecution> getAllProcessStates()
+ {
+ return processStateList;
+ }
+
+ /**
+ * Returns a collection of information on all running processes.
+ * @return Returns a collection of information on all running processes. The returned result should not be modified.
+ */
+ synchronized Collection<ProcessletExecution> getRunningProcessStates()
+ {
+ return responseDocumentIdToState.values();
+ }

/**
* Handles {@link ExecuteRequest} requests that shall return a single "raw" output param (with no encapsulating
@@ -164,7 +204,7 @@
}
ProcessletOutputs outputParams = new ProcessletOutputs( processDef, Collections.singletonList( outputParam ) );

- ProcessletExecution state = new ProcessletExecution( request, null, null, null, outputParams );
+ ProcessletExecution state = createProcessletExecution( request, null, null, null, outputParams );
executeProcess( process, inputs, outputParams, state );
}

@@ -235,8 +275,10 @@

// allocate a storage location for the final response document
responseStorage = storageManager.allocateResponseDocumentStorage();
-
- state = new ProcessletExecution( request, responseStorage, serviceInstance, outputParams, outputs );
+
+ state = createProcessletExecution( request, responseStorage, serviceInstance, outputParams, outputs );
+
+ state.setResponseStorageUrl(new URL(responseStorage.getWebURL()));

// submit the process for asynchronous execution
ProcessWorker worker = new ProcessWorker( process, outputs, state, outputParams, responseStorage, request );
@@ -250,7 +292,7 @@
throw new OWSException( msg, OWSException.INVALID_PARAMETER_VALUE, "status" );
}

- state = new ProcessletExecution( request, responseStorage, serviceInstance, outputParams, outputs );
+ state = createProcessletExecution( request, responseStorage, serviceInstance, outputParams, outputs );
executeProcess( process, inputs, outputs, state );
}

@@ -413,7 +455,7 @@

// register the storage location of the response document from the lookup map
responseDocumentIdToState.put( responseStorage, state );
-
+
executeProcess( process, request.getDataInputs(), outputs, state );

LOG.debug( "Storing final response document at " + responseStorage );
Index: src/org/deegree/services/controller/wps/ProcessletExecution.java
===================================================================
--- src/org/deegree/services/controller/wps/ProcessletExecution.java (revision 19222)
+++ src/org/deegree/services/controller/wps/ProcessletExecution.java (working copy)
@@ -40,6 +40,7 @@
import java.util.Date;
import java.util.List;

+import org.deegree.commons.types.ows.CodeType;
import org.deegree.commons.utils.time.DateUtils;
import org.deegree.services.controller.ows.OWSException;
import org.deegree.services.controller.wps.execute.ExecuteRequest;
@@ -82,6 +83,8 @@
private int percentCompleted;

private StorageLocation responseStorage;
+
+ private URL responseStorageUrl = null;

private URL serviceInstance;

@@ -155,6 +158,14 @@
}

/**
+ * Returns optional response storage URL.
+ * @return Returns optional response storage URL, if there is no response storage URL a null is returned.
+ */
+ public URL getResponseStorageUrl() {
+ return responseStorageUrl;
+ }
+
+ /**
* Returns optional additional human-readable text associated with the starting of the process execution.
*
* @return optional additional human-readable text, null if it is not available
@@ -185,6 +196,14 @@
public long getFinishTime() {
return finishTime;
}
+
+ /**
+ * Returns the "Identifier" for this process.
+ * @return Returns the CodeType or "Identifier" for this process.
+ */
+ public CodeType getProcessId() {
+ return request.getProcessId();
+ }

/**
* Returns optional additional human-readable text associated with the pausing of the process execution.
@@ -202,7 +221,7 @@
* @return optional additional human-readable text, null if it is not available
*/
public String getSucceededMessage() {
- if ( succeededMessage == null ) {
+ if ( succeededMessage == null && finishTime >= 0) {
return "Process execution succeeded@" + DateUtils.formatISO8601Date( new Date( finishTime ) );
}
return succeededMessage;
@@ -267,6 +286,15 @@
public ExecuteResponse createExecuteResponse() {
return new ExecuteResponse( responseStorage, serviceInstance, this, outputParams, outputs, request );
}
+
+ /**
+ * Sets the response storage web URL. This URL may be queried to get this process' status.
+ *
+ * @return a response storage web URL that reflects the current execution state
+ */
+ public void setResponseStorageUrl( URL url ) {
+ this.responseStorageUrl = url;
+ }

// -----------------------------------------------------------------------
// methods from ProcessletExecutionInfo
Index: src/org/deegree/services/controller/wps/WPSController.java
===================================================================
--- src/org/deegree/services/controller/wps/WPSController.java (revision 19222)
+++ src/org/deegree/services/controller/wps/WPSController.java (working copy)
@@ -83,9 +83,9 @@
import org.deegree.protocol.wps.describeprocess.DescribeProcessRequest;
import org.deegree.protocol.wps.describeprocess.DescribeProcessRequestKVPAdapter;
import org.deegree.protocol.wps.describeprocess.DescribeProcessRequestXMLAdapter;
+import org.deegree.services.controller.AbstractOGCServiceController;
import org.deegree.services.controller.ImplementationMetadata;
import org.deegree.services.controller.OGCFrontController;
-import org.deegree.services.controller.AbstractOGCServiceController;
import org.deegree.services.controller.configuration.DeegreeServicesMetadata;
import org.deegree.services.controller.exception.ControllerException;
import org.deegree.services.controller.exception.ControllerInitException;
@@ -214,9 +214,9 @@
// WSDL stuff
serviceWSDLFile = FileUtils.getAsFile( new URL( controllerConfURL, "../service.wsdl" ) );
} catch ( MalformedURLException e ) {
- throw new ControllerInitException( "Problem resolving file resource: " + e.getMessage() );
+ throw new ControllerInitException( "Problem resolving file resource: " + e.getMessage(), e );
} catch ( ServiceInitException e ) {
- throw new ControllerInitException( "Problem initializing service: " + e.getMessage() );
+ throw new ControllerInitException( "Problem initializing service: " + e.getMessage(), e );
}
}

@@ -431,6 +431,16 @@
LOG.debug( e.getMessage(), e );
}
}
+
+ public Collection<ProcessletExecution> getAllProcessStates()
+ {
+ return executeHandler.getAllProcessStates();
+ }
+
+ public Collection<ProcessletExecution> getRunningProcessStates()
+ {
+ return executeHandler.getAllProcessStates();
+ }

private WPSRequestType getRequestTypeByName( String requestName )
throws OWSException {

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
deegree-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/deegree-devel

Jason R. Surratt

Re: Status on all running Processes

Reply Threaded

Hi Marcus,

Attached is simple unit test patch for listing processes -- it should at least catch fundamental problems. I had to hack a little in OGCFrontController to expose the necessary bits for testing, but the hack is fairly benign and not exposed to the public interface.

Cheers!

-jason

On Tue, Aug 18, 2009 at 7:21 PM, Jason R. Surratt <[hidden email]> wrote:

> Good call. I've updated the wiki with those changes.
>
> Should I just post a patch to the mailing list when I've got a first cut
> of my changes?

Yes, just send it to list. When this gets too cumbersome, we can set up an SVN commit account for you.

Marcus,

The patch is attached. I've highlighted some things of note below.
I didn't see any straight forward way to get an instance of OGCFrontController from the .jsp files. I've added a quick hack in for now that should definitely be removed. It appears that ServletContext#getServlet used to provide this functionality but it has since been deprecated. My suggestion is to refactor OGCFrontController into two classes, a singleton that provides the bulk of the functionality and a servlet. However, I didn't feel comfortable hacking and slashing Deegree that much w/o mentioning it first.

It appears that the "responseDocumentIdToState" HashMap could have suffered from concurrency issues within Processworker#run, I've converted it to a ConcurrentHashMap to avoid the problem.
I am ashamed to say I didn't add any unit tests in yet, but I'll get that done tomorrow. Due to the time difference I wanted to make sure I got something off to you tonight.

I've included two .jsp files, one that lists the processes in as very simple html and another that response with simple XML (the functionality I need)
Please let me know what you think. Thanks!

-jason

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
deegree-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/deegree-devel

Markus Schneider-4

Re: Status on all running Processes

Reply Threaded

Hi Jason,

first let me say thank you again for submitting the patches. I already added some parts of it to the SVN, as you already
recognized. The main change that I performed is that I restricted the number of process executions that the
ExecutionManager stores -- otherwise this list will eventually become to large and exceed the available heap memory. As
we already discussed, the perfect solution would be to use a database here.

Does the current state provide enough functionality for your use case (i.e. the XML process listing)?

Best regards
Markus

Jason R. Surratt wrote:

> Hi Marcus,
>
> Attached is simple unit test patch for listing processes -- it should at
> least catch fundamental problems. I had to hack a little in
> OGCFrontController to expose the necessary bits for testing, but the
> hack is fairly benign and not exposed to the public interface.
>
> Cheers!
>
> -jason
>
> On Tue, Aug 18, 2009 at 7:21 PM, Jason R. Surratt
> <[hidden email] <mailto:[hidden email]>> wrote:
>
>
> > Good call. I've updated the wiki with those changes.
> >
> > Should I just post a patch to the mailing list when I've got a
> first cut
> > of my changes?
>
> Yes, just send it to list. When this gets too cumbersome, we can
> set up an SVN commit account for you.
>
>
> Marcus,
>
> The patch is attached. I've highlighted some things of note below.
>
> * I didn't see any straight forward way to get an instance of
> OGCFrontController from the .jsp files. I've added a quick
> hack in for now that should definitely be removed. It appears
> that ServletContext#getServlet used to provide this
> functionality but it has since been deprecated. My suggestion
> is to refactor OGCFrontController into two classes, a
> singleton that provides the bulk of the functionality and a
> servlet. However, I didn't feel comfortable hacking and
> slashing Deegree that much w/o mentioning it first.
> * It appears that the "responseDocumentIdToState" HashMap could
> have suffered from concurrency issues within
> Processworker#run, I've converted it to a ConcurrentHashMap to
> avoid the problem.
> * I am ashamed to say I didn't add any unit tests in yet, but
> I'll get that done tomorrow. Due to the time difference I
> wanted to make sure I got something off to you tonight.
> * I've included two .jsp files, one that lists the processes in
> as very simple html and another that response with simple XML
> (the functionality I need)
>
> Please let me know what you think. Thanks!
>
> -jason
>
>
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now. http://p.sf.net/sfu/bobj-july
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> deegree-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/deegree-devel

signature.asc (268 bytes) Download Attachment

Jason R. Surratt

Re: Status on all running Processes

Reply Threaded

Hi Marcus,

first let me say thank you again for submitting the patches.

My pleasure. :)

I already added some parts of it to the SVN, as you already
recognized. The main change that I performed is that I restricted the number of process executions that the
ExecutionManager stores -- otherwise this list will eventually become to large and exceed the available heap memory. As
we already discussed, the perfect solution would be to use a database here.

Makes sense to me.

Does the current state provide enough functionality for your use case (i.e. the XML process listing)?

Mostly. I noticed you removed ProcessExecution#getResponseStorageUrl() and associated code from the initial patch. Unfortunately, without this or something similar I don't think there is any way for me to get back to the detailed process response (if serialized).

To give you a little more background my use case goes like this:

* User specifies one or more very large raster operations and launches them through a flex interface, which calls WPS
* The user can then see status as the job progresses via flex, (Calls to my custom servlet returning the status XML)
* <go home for the night>
* User goes to the progress screen and checks out the status and results of yesterday's run. The response contains a link to WMS, KML or equivalent for viewing the result.

Thanks again for all your help!

-jason

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
deegree-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/deegree-devel

Markus Schneider-4

Re: Status on all running Processes

Reply Threaded

Hi Jason,

I am sorry that I didn't have the time to deal with your remarks in detail, but I am a bit in a hassle at the moment,
because I am about to embark on summer holidays. I hope this doesn't give you any troubles getting your WPS project
finished.

I will reply to the outstanding questions when I am back, which will be on September 21.

Best regards,
Markus

Jason R. Surratt wrote:

> Hi Marcus,
>
> first let me say thank you again for submitting the patches.
>
>
> My pleasure. :)
>
>
> I already added some parts of it to the SVN, as you already
> recognized. The main change that I performed is that I restricted
> the number of process executions that the
> ExecutionManager stores -- otherwise this list will eventually
> become to large and exceed the available heap memory. As
> we already discussed, the perfect solution would be to use a
> database here.
>
>
> Makes sense to me.
>
>
> Does the current state provide enough functionality for your use
> case (i.e. the XML process listing)?
>
>
> Mostly. I noticed you removed ProcessExecution#getResponseStorageUrl()
> and associated code from the initial patch. Unfortunately, without this
> or something similar I don't think there is any way for me to get back
> to the detailed process response (if serialized).
>
> To give you a little more background my use case goes like this:
>
> * User specifies one or more very large raster operations and launches
> them through a flex interface, which calls WPS
> * The user can then see status as the job progresses via flex, (Calls
> to my custom servlet returning the status XML)
> * <go home for the night>
> * User goes to the progress screen and checks out the status and
> results of yesterday's run. The response contains a link to WMS, KML or
> equivalent for viewing the result.
>
> Thanks again for all your help!
>
> -jason
>
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now. http://p.sf.net/sfu/bobj-july
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> deegree-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/deegree-devel

signature.asc (268 bytes) Download Attachment

Jason R. Surratt

Re: Status on all running Processes

Reply Threaded

Markus,

No worries. I figured you were busy. I've made enough modifications to my local copy of the code to keep going. When you get back I'd be happy to pick up were we left off -- I'd rather not perpetually maintain my mods, especially if they will be helpful to someone else.

Enjoy your vacation!

-jason

On Fri, Aug 28, 2009 at 10:43 AM, Markus Schneider <[hidden email]> wrote:

Hi Jason,

I am sorry that I didn't have the time to deal with your remarks in detail, but I am a bit in a hassle at the moment,
because I am about to embark on summer holidays. I hope this doesn't give you any troubles getting your WPS project
finished.

I will reply to the outstanding questions when I am back, which will be on September 21.

Best regards,
Markus

Jason R. Surratt wrote:

> Hi Marcus,
>
> first let me say thank you again for submitting the patches.
>
>
> My pleasure. :)
>
>
> I already added some parts of it to the SVN, as you already
> recognized. The main change that I performed is that I restricted
> the number of process executions that the
> ExecutionManager stores -- otherwise this list will eventually
> become to large and exceed the available heap memory. As
> we already discussed, the perfect solution would be to use a
> database here.
>
>
> Makes sense to me.
>
>
> Does the current state provide enough functionality for your use
> case (i.e. the XML process listing)?
>
>
> Mostly. I noticed you removed ProcessExecution#getResponseStorageUrl()
> and associated code from the initial patch. Unfortunately, without this
> or something similar I don't think there is any way for me to get back
> to the detailed process response (if serialized).
>
> To give you a little more background my use case goes like this:
>
> * User specifies one or more very large raster operations and launches
> them through a flex interface, which calls WPS
> * The user can then see status as the job progresses via flex, (Calls
> to my custom servlet returning the status XML)
> * <go home for the night>
> * User goes to the progress screen and checks out the status and
> results of yesterday's run. The response contains a link to WMS, KML or
> equivalent for viewing the result.
>
> Thanks again for all your help!
>
> -jason
>
>

> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now. http://p.sf.net/sfu/bobj-july
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> deegree-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/deegree-devel

--
Markus Schneider

l a t / l o n GmbH
Aennchenstrasse 19 53177 Bonn, Germany
phone ++49 +228 184960 fax ++49 +228 1849629
http://www.lat-lon.de http://www.deegree.org

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
deegree-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/deegree-devel