How do I create a new Webserivce for Sonar?

[Christopher Moraes]

Hi Everyone,

I am working on a "Code Review Plugin" for sonar, which will allow a user to view sources in the Violations Drilldown view and add review comments to the sources.

I have created a new "Code Review" tab which is similar to the "Violations" (source viewer) tab.  A user clicks on the source line number and a popup window appears in which the user can enter their comments.  Once the user is done there is a button that should submit the comments back to the server.

I need to send this 'consolidated comments' object (which is a List of comment objects) back to the server, where a webservice will pick it up for further processing.  

Can someone please explain how I should create a new webservice for sonar.

Should I write a new webservice similar to the ones that already exist under sonar-web/src/main/webapp/WEB-INF/app/controllers/api and drop it in the <sonar_home>/extensions/plugins directory?

Alternatively can I write a webservice without using the RoR framework.  I do not know Ruby, so i would prefer to write this in Java if possible.

Regards,

Chris

[Simon Brandhof]

Hi Christopher,

Your idea to add code reviews to Sonar is really good.
Implementing a web service is possible and rather easy, even if it's written in Ruby. But the real problem is that I don't know where data will be saved. None of the current database tables match the model you need.

The only solution is to develop a patch for Sonar core. We can imagine a new table SOURCE_FLAGS, in which each item decorates a resource (class/package/project) or a block of code lines. Columns would be something like resource_id, nature ('review', ...) , from_line, to_line, data, ... This table is accessed via a Java API and a Web Service.

I don't think that it's too much difficult. Are you motivated to give a try ? Of course you're not alone, you'll benefit from all our advices and help.

Regards,
Simon

[Christopher Moraes]

Hi Simon,

Thanks for your suggestion.  

But the real problem is that I don't know where data will be saved. None of the current database tables match the model you need.

The review comments will be inserted back into the source code in the SVN repository. This is a requirement that I have, because the organization for whom I am developing this, wants to keep the comments and the code tied together tightly.  I am planning to use the java SVNKit library to insert the comments received back into the source code.  

In order to send the comment objects back to the server I am considering using GWT RPC with a GWT servlet on the server side.  The servlet will be able to easily call the java svnkit library.  

What do you think of this approach?  

 

The only solution is to develop a patch for Sonar core. We can imagine a new table SOURCE_FLAGS, in which each item decorates a resource (class/package/project) or a block of code lines. Columns would be something like resource_id, nature ('review', ...) , from_line, to_line, data, ... This table is accessed via a Java API and a Web Service.
 

I don't think that it's too much difficult. Are you motivated to give a try ? Of course you're not alone, you'll benefit from all our advices and help.

Thanks for this suggestion.  I had not considered writing a patch to Sonar core before.  If I can store the review comments in the database, then this plugin can develop into a code review sub-system that will be able to track review comments and responses. I will definitely look at this option.  

 

Regards,
Simon

[Freddy Mallet]

Hi Christopher,

Do you already know the format you're going to use in order to insert review comments in the source code ? I'm a bit curious about that :-)

regards,
Freddy

On Wed, Oct 21, 2009 at 5:53 PM, Christopher Moraes <[hidden email]> wrote:

Hi Simon,

Thanks for your suggestion.  

But the real problem is that I don't know where data will be saved. None of the current database tables match the model you need.

The review comments will be inserted back into the source code in the SVN repository. This is a requirement that I have, because the organization for whom I am developing this, wants to keep the comments and the code tied together tightly.  I am planning to use the java SVNKit library to insert the comments received back into the source code.  

In order to send the comment objects back to the server I am considering using GWT RPC with a GWT servlet on the server side.  The servlet will be able to easily call the java svnkit library.  

What do you think of this approach?  

 

The only solution is to develop a patch for Sonar core. We can imagine a new table SOURCE_FLAGS, in which each item decorates a resource (class/package/project) or a block of code lines. Columns would be something like resource_id, nature ('review', ...) , from_line, to_line, data, ... This table is accessed via a Java API and a Web Service.
 

I don't think that it's too much difficult. Are you motivated to give a try ? Of course you're not alone, you'll benefit from all our advices and help.

Thanks for this suggestion.  I had not considered writing a patch to Sonar core before.  If I can store the review comments in the database, then this plugin can develop into a code review sub-system that will be able to track review comments and responses. I will definitely look at this option.  

 

Regards,
Simon

[Christopher Moraes]

Hi Freddy,

Do you already know the format you're going to use in order to insert review comments in the source code ? I'm a bit curious about that :-)

I'm using the following format to create a comment - that is identified as a review comment.

   /*

    * @SonarReviewComment 

         @Reviewer        Christopher

         @CommentAssignee Development

         @ReviewDate      10-Oct-2008

         @ActionExpected  Fix

    *

         This is a test Comment 

         This is a test Comment 

         This is a test Comment 

    */

Also, the following comment will be inserted at the beginning of the file to flag this file as being "code reviewed".  

/* 

 * @SonarFileReview reviewer=Christopher Date=10-Oct-09 svnVersion=82

 */

I am intentionally not using javadoc comments right now, as I don't want these comments to be processed by a javadoc tool.   

In order to send the comment objects back to the server I am considering using GWT RPC with a GWT servlet on the server side.  The servlet will be able to easily call the java svnkit library.  

I have managed to get this part working and now I am able to send the comments back to the server and print them out there.  

I realised that in order to update the repository I need to obtain the svn url of each file in the source code, and store it in Sonar somewhere.  I am thinking of capturing the svn url and version number using a sensor.

Could you suggest the where I could store this information in the Sonar database and how I could create new tables/table attributes in Sonar?

 

Regards,

Chris

[Simon Brandhof]

Hi Christopher,

Here are some answers to your previous emails :

• Plugins can not define their own classpath (no OSGi for the moment). The workaround to use svnkit is to add the JAR in extensions/plugins
• Only the GWT generated javascript is supported, not server-side components (servlets/RPC). I'm afraid the only solution is to develop a JRuby on Rails web service...  Have a look at the plugin radiator to get a full sample.
  
• A measure with text value can be used to save Subversion URLs. Just define a metric named like 'scm_url' and, like you said, inject the measure from a sensor.
• Plugins can not modify the database structure. To do it in core, you have to follow many steps in Ruby and Java. Rather than listing all of them (not so easy if you don't know ruby), I suggest to discuss now of your requirements (columns, indexes). Then we will create the table for you in a future release. Of course this table must be reusable to save any kind of data on files.

Regards,
Simon

[Christopher Moraes]

Hi Simon,

• A measure with text value can be used to save Subversion URLs. Just define a metric named like 'scm_url' and, like you said, inject the measure from a sensor.

Can I use this Metric at a source file level or is this only one metric per project?  I would like to store the svn checksum and svn url for each file in the source tree.

Regards,

Chris

 

[Simon Brandhof]

Can I use this Metric at a source file level or is this only one metric per project?  I would like to store the svn checksum and svn url for each file in the source tree.

Indeed you can save this measure on files only.

Example :
Metric SCM_URL = new Metric("scm_url", "SCM URL", "SCM URL", Metric.ValueType.DATA, Metric.DIRECTION_NONE, false, null);

measure=new Measure(SCM_URL, "svn:....");
sensorContext.saveMeasure(myFile, measure);

[Christopher Moraes]

Hi Simon,

I am working on this sonar plugin which I want to collect the svn url and svn checksum for each java source file in the project and store it as a measure in sonar. I have a question about how to use the Sensor extension point.

My project consists of 3 parts 

1.  An application called "svninfo".

This standalone java application traverses the directory of an svn working copy  and generates an "svninfo.xml" file with details of each file in the directory.

The output xml file has the following format

<info>

   <entry>

      <kind>file</kind>

      <path>/home/cmoraes/code/dev/sonar/test-project-sonar/src/main/java/com/symcor/sonar/test/SVNTest.java</path>

      <url>svn://localhost/repos/repo1/trunk/test-project-sonar/src/main/java/com/symcor/sonar/test/SVNTest.java</url>

      <checksum>42e6d5c210e09a4f9f3206da750f068c</checksum>

      <rev>14</rev>

   </entry>

   <entry>

      <kind>file</kind>

      <path>/home/cmoraes/code/dev/sonar/test-project-sonar/src/main/java/com/symcor/sonar/test/Sample.java</path>

      <url>svn://localhost/repos/repo1/trunk/test-project-sonar/src/main/java/com/symcor/sonar/test/Sample.java</url>

      <checksum>63d34b14bb2b093a7202cbcf4952208c</checksum>

      <rev>14</rev>

   </entry>

</info>

2.  A "maven-svninfo-plugin" - that invokes the svninfo application with the project source directory as the parameter.  This plugin is invoked as "mvn svninfo:svninfo".

3.  The sonar-plugin-svninfo.   

I want this plugin to invoke the maven-svinfo-plugin, read the output xml file generated, and then store the svn url and checksum for each source file in the project - in a measure.

I have gone through the sonar sources for checkstyle and squid, and I have created the 

i)  Plugin class

ii) MavenPluginHandler class

I'm not sure what should be done in the Sensor class.  I know I can read the svninfo.xml file in the analyse() method, but am not sure how I should store these as measures, as you've suggested below.  I guess I need one measure per file (in the source)?

In the example you gave below, is the parameter "myfile" any string that identifies a file?   Would the following be correct ?

myfile = "com.symcor.sonar.test.Sample.java";

Would appreciate if you could explain this.

Thanks and regards,

Chris

On Wed, Oct 28, 2009 at 8:19 AM, Simon Brandhof <[hidden email]> wrote:

Can I use this Metric at a source file level or is this only one metric per project?  I would like to store the svn checksum and svn url for each file in the source tree.

Indeed you can save this measure on files only.

Example :
Metric SCM_URL = new Metric("scm_url", "SCM URL", "SCM URL", Metric.ValueType.DATA, Metric.DIRECTION_NONE, false, null);

measure=new Measure(SCM_URL, "svn:....");
sensorContext.saveMeasure(myFile, measure);

[Olivier Gaudin-2]

Hello Christopher,

I think you should find great inspiration in the Sonar Taglist plugin as the mechanism is very similar : http://svn.codehaus.org/sonar-plugins/trunk/taglist/src/main/java/org/sonar/plugins/taglist/.
The entry point is the sensor, but what you are interested in is the parser.

From your email, I see that you are gonna have to transform absolute path to relative path. Note that there is a hole bunch of utility to do this in the API.

Olivier

On Wed, Nov 4, 2009 at 4:56 AM, Christopher Moraes <[hidden email]> wrote:

Hi Simon,

I am working on this sonar plugin which I want to collect the svn url and svn checksum for each java source file in the project and store it as a measure in sonar. I have a question about how to use the Sensor extension point.

My project consists of 3 parts 

1.  An application called "svninfo".

This standalone java application traverses the directory of an svn working copy  and generates an "svninfo.xml" file with details of each file in the directory.

The output xml file has the following format

<info>

   <entry>

      <kind>file</kind>

      <path>/home/cmoraes/code/dev/sonar/test-project-sonar/src/main/java/com/symcor/sonar/test/SVNTest.java</path>

      <url>svn://localhost/repos/repo1/trunk/test-project-sonar/src/main/java/com/symcor/sonar/test/SVNTest.java</url>

      <checksum>42e6d5c210e09a4f9f3206da750f068c</checksum>

      <rev>14</rev>

   </entry>

   <entry>

      <kind>file</kind>

      <path>/home/cmoraes/code/dev/sonar/test-project-sonar/src/main/java/com/symcor/sonar/test/Sample.java</path>

      <url>svn://localhost/repos/repo1/trunk/test-project-sonar/src/main/java/com/symcor/sonar/test/Sample.java</url>

      <checksum>63d34b14bb2b093a7202cbcf4952208c</checksum>

      <rev>14</rev>

   </entry>

</info>

2.  A "maven-svninfo-plugin" - that invokes the svninfo application with the project source directory as the parameter.  This plugin is invoked as "mvn svninfo:svninfo".

3.  The sonar-plugin-svninfo.   

I want this plugin to invoke the maven-svinfo-plugin, read the output xml file generated, and then store the svn url and checksum for each source file in the project - in a measure.

I have gone through the sonar sources for checkstyle and squid, and I have created the 

i)  Plugin class

ii) MavenPluginHandler class

I'm not sure what should be done in the Sensor class.  I know I can read the svninfo.xml file in the analyse() method, but am not sure how I should store these as measures, as you've suggested below.  I guess I need one measure per file (in the source)?

In the example you gave below, is the parameter "myfile" any string that identifies a file?   Would the following be correct ?

myfile = "com.symcor.sonar.test.Sample.java";

Would appreciate if you could explain this.

Thanks and regards,

Chris

On Wed, Oct 28, 2009 at 8:19 AM, Simon Brandhof <[hidden email]> wrote:

Can I use this Metric at a source file level or is this only one metric per project?  I would like to store the svn checksum and svn url for each file in the source tree.

Indeed you can save this measure on files only.

Example :
Metric SCM_URL = new Metric("scm_url", "SCM URL", "SCM URL", Metric.ValueType.DATA, Metric.DIRECTION_NONE, false, null);

measure=new Measure(SCM_URL, "svn:....");
sensorContext.saveMeasure(myFile, measure);

[Christopher Moraes]

Hi Oliver,

Thanks for that link.  I have completed my sonar-svninfo-plugin and have run it successfully (I think) against a test project.   

Is there an easy way to confirm that the measures have actually been loaded into the sonar DB?  (I am using the embedded derby database)

Also, now that the measures are in Sonar - could you give me a pointer how the measure should be retrieved from within a gwt client component.   Would it be right to use the "ResourcesQuery" class to retrieve the measures?

Thanks and regards,

Chris

On Wed, Nov 4, 2009 at 3:26 AM, Olivier Gaudin <[hidden email]> wrote:

Hello Christopher,

I think you should find great inspiration in the Sonar Taglist plugin as the mechanism is very similar : http://svn.codehaus.org/sonar-plugins/trunk/taglist/src/main/java/org/sonar/plugins/taglist/.
The entry point is the sensor, but what you are interested in is the parser.

From your email, I see that you are gonna have to transform absolute path to relative path. Note that there is a hole bunch of utility to do this in the API.

Olivier

[Simon Brandhof]

Is there an easy way to confirm that the measures have actually been loaded into the sonar DB?  (I am using the embedded derby database)

The easiest way is to use a database like MySql or Postgresql.
 

Also, now that the measures are in Sonar - could you give me a pointer how the measure should be retrieved from within a gwt client component.   Would it be right to use the "ResourcesQuery" class to retrieve the measures?

 

Indeed the class ResourcesQuery is a GWT client for the webservice that loads measures. Here is an example of usage :
http://svn.codehaus.org/sonar/trunk/plugins/sonar-plugin-core/src/main/java/org/sonar/plugins/core/hotspots/client/widget/MostViolatedResources.java

Regards
Simon

[Christopher Moraes]

Simon, Oliver,

Thanks for all your inputs.  I was able to get this working and now am able to retrieve the measure in my GWT component.

Regards,

Chris