Maven2 POM to CPEs

[Ross Jekel]

Hello everyone,

I'm new to the list, but I have talked to Drew recently on the phone. Hi Drew! Thank you all for working on this effort to standardize names.

I work for SourceLabs, Inc, a company that is actively involved in both contributing to and supporting many open source Java frameworks, packages, and libraries. We have an immediate need for enumerating these packages. Paul Whyman also mentioned a couple of days ago that he is also interested in open source package naming, so I thought I would chime in now with a modest proposal.

SourceLabs Inc is interested in helping create CPEs for all Java libraries in the Maven2 repositories (http://repo1.maven.org/maven2). While not all Java Apps/Libraries use Maven2's POM to describe the project, most of the popular ones do. We believe the POM format (http://maven.apache.org/pom.html) has sufficient required information to automatically generate meaningful and unique CPEs for Java libraries in the repository. There have already been security bulletins against some of the software we support such as struts (e.g. http://nvd.nist.gov/nvd.cfm?cvename=CVE-2006-1546), the POM file for which one can easily find in the Maven2 repository here (http://repo1.maven.org/maven2/struts/struts/1.2.8/struts-1.2.8.pom). Since there is a CVE against struts, I assume a name will eventually be assigned, but we have an interest in developing CPEs for java products even if they have not yet had a security bulletin against them.

If you aren't already familiar with POM files, a couple more examples might be useful. A simple one can be seen here:

http://repo1.maven.org/maven2/axis/axis-wsdl4j/1.5.1/axis-wsdl4j-1.5.1.pom (as you can see, groupId, artifactId, and version are the minimum we have to work with)

and a more complex example would be:

http://repo1.maven.org/maven2/org/mortbay/jetty/jetty/4.2.12/jetty-4.2.12.pom

I think together we can come up with an acceptable mapping from the POM groupId, artifactId, and version data to CPE names that can be used to automatically generate CPEs, then I could write code here that could periodically look for new POM files in our mirror and generate CPE names to be sent to Mitre for inclusion in the dictionary. Preferably the mapping algorithm would be able to go both directions (POM <-> CPE), but I suppose we could put the backlink to the POM file in the notes field if this is not possible given the current CPE rules. Since this effort would result in a large number of new CPEs (I have almost 22000 POM files in my current mirror), I thought it best to include everyone in the original design and decision making process.

Since the minimum information (groupId, artifactId, and version) does not necessarily follow CPE naming rules but is guaranteed unique in the Maven2 repository namespace, one suggestion that helps keep the mapping algorithm simple is to have a relaxed set of rules for generating CPEs from POM files under a new "part" letter ('p' might be good for "POM" or "Package"). Most items in Maven2 aren't stand-alone apps (an 'a' part), so a new letter might be appropriate.

Alternatively, we may introduce a more generic 'l' part for library (possibly needed by things like glibc which I know has historical CVEs against it), and somehow figure out how to distinguish POM CPEs within that part designation.

If we can collectively decide on an direction, I can write up a proposal for inclusion in the specification and begin the work of creating the names for the dictionary.

Thanks,
Ross

[Andrew Buttner]

Ross,

Again, thank you very much for your interest in CPE.  Help from the
community is what will make this effort a success!

>I think together we can come up with an acceptable mapping
>from the POM groupId, artifactId, and version data to CPE
>names that can be used to automatically generate CPEs, then I
>could write code here that could periodically look for new POM
>files in our mirror and generate CPE names to be sent to Mitre
>for inclusion in the dictionary.

Maybe a good start would be to post a few examples with the specific
Java package and the proposed CPE Name.


>Since the minimum information (groupId, artifactId, and
>version) does not necessarily follow CPE naming rules but is
>guaranteed unique in the Maven2 repository namespace, one
>suggestion that helps keep the mapping algorithm simple is to
>have a relaxed set of rules for generating CPEs from POM files
>under a new "part" letter ('p' might be good for "POM" or
>"Package"). Most items in Maven2 aren't stand-alone apps (an
>'a' part), so a new letter might be appropriate.

Can you elaborate on how the POM info does not follow the CPE naming
rules?  Is this something that could be taken care of with some
scripting?


>Alternatively, we may introduce a more generic 'l' part for
>library (possibly needed by things like glibc which I know has
>historical CVEs against it), and somehow figure out how to
>distinguish POM CPEs within that part designation.

The idea of naming libraries is an interesting one.  At first glance I
would think: absolutely.  But thinking a little deeper, libraries are
different then the current applications, operating systems, and
hardware parts that we current have.  In the past, I have always
thought of a CPE Name as identifying a platform that has the named
app/os/harware installed.  But with libraries, you don't have a
platform with the library installed, instead you might have a platform
with an application installed that uses the library.  Subtle but
interesting difference.  I probably just need to change the way I think
about CPE.

What do others think?

[Ken Lassesen-3]

My 2 cents:

I am very much in favor of decomposing anything that runs on an OS (and
some of the OS Components!) into their components (at least to support
it!).  Many CVE's apply to many products because they all happen to use
the same library or dll. I would suggest
* "l" for libraries (including Controls (OCX's) other code bases that
are not directly executed from the OS (aka .exe)
* "j" for Java -- I feel we need to break this out into it's own
namespace
* "e" for executable

So within the CPL (Language) a CPE for an application ("a") may be
broken down into "l","e" and "j" parts.  A CVE may then be more
precisely identified to the specific element of an application.

As a FYI: I'm very much in favor of following Joe's suggestion of having
CPE being a number with the CPL containing the multitude of metadata
that we are attempting to overload the CPE ID with. Going on a crusade
to find the holy grail is nice and noble --- but we need to put ploughs
to the soil NOW. Put off the quest and let us get something out now that
gives us a clean ID with the metadata being separate (and likely verbose
in many cases).


Ken Lassesen,
Office 206-734-4718 Home: 360-297-4717   Cell: 360-509-2402  Skype:
Ken.Lassesen
IM: [hidden email]  

CONFIDENTIALITY NOTICE
The information contained in this electronic message may contain
confidential and privileged information and is intended only for use by
the individual(s) or entity(ies) to whom it was addressed. Any
unauthorized review, use, disclosure, or distribution of this
communication is strictly prohibited. If you are not the intended
recipient, please contact the sender by reply email and permanently
delete and destroy the original message.

[Andrew Buttner]

>I am very much in favor of decomposing anything that runs on an OS
(and
>some of the OS Components!) into their components (at least to support
>it!).  Many CVE's apply to many products because they all happen to
use
>the same library or dll. I would suggest
>* "l" for libraries (including Controls (OCX's) other code bases that
>are not directly executed from the OS (aka .exe)
>* "j" for Java -- I feel we need to break this out into it's own
>namespace
>* "e" for executable

Here is a question that has been bothering me about this ... How would
we write an OVAL Definition to check for an "L" (library) CPE Name?  If
we could answer this I would be very comfortable with CPE Names for
libraries.

Drew

[Ken Lassesen-3]

The checking method depends on whether the installion program followed
"appropriate" guidance. For shareware and some 'commercial' products,
this can be a major stretch. For a while there was a practise of
installing 'system' dll's in the application install folder because the
next version was *not* backward compatible and this was the easy
solution.

The most conservative solution is to do a full drive search. A well
design product would cache the first scan and use it for all subsequent
lookup for dll's. If the tool implements a file system watcher, then the
full list of libraries just needs to be loaded from the disk-cache file.

Where dll's are shared, they typically end up in:
* C:\WINDOWS\system32 (and subfolders) especially
C:\WINDOWS\system32\dllcache
* C:\WINDOWS\Microsoft.Net (and subfolders)
* C:\WINDOWS\Downloaded Program Files (and subfolders)
* C:\Program Files\Common Files (and subfolders)

Not perfect solution, but as we all know, the nature of a human process
is usually chaos.

Ken Lassesen,
Office 206-734-4718 Home: 360-297-4717   Cell: 360-509-2402  Skype:
Ken.Lassesen
IM: [hidden email]  

CONFIDENTIALITY NOTICE
The information contained in this electronic message may contain
confidential and privileged information and is intended only for use by
the individual(s) or entity(ies) to whom it was addressed. Any
unauthorized review, use, disclosure, or distribution of this
communication is strictly prohibited. If you are not the intended
recipient, please contact the sender by reply email and permanently
delete and destroy the original message.


-----Original Message-----
From: Buttner, Drew [mailto:[hidden email]]
Sent: Monday, October 08, 2007 12:54 PM
To: [hidden email]
Subject: Re: [CPE-DISCUSSION-LIST] Finer resolution in CPE.

>I am very much in favor of decomposing anything that runs on an OS
(and
>some of the OS Components!) into their components (at least to support
>it!).  Many CVE's apply to many products because they all happen to
use
>the same library or dll. I would suggest
>* "l" for libraries (including Controls (OCX's) other code bases that
>are not directly executed from the OS (aka .exe)
>* "j" for Java -- I feel we need to break this out into it's own
>namespace
>* "e" for executable

Here is a question that has been bothering me about this ... How would
we write an OVAL Definition to check for an "L" (library) CPE Name?  If
we could answer this I would be very comfortable with CPE Names for
libraries.

Drew

[Vladimir Giszpenc]

Hi Ken et al,

Granularity is always desired, but Java is not meaningful enough.  Now, if
you mean the JRE, I would counter that there are other runtime environments.
Virtualization as a whole should be addressed.  You have it at the OS level
with VMWare, hypervisors and hardware solutions.  This creates the need to
define hosts, guests and a dom0 which (I think) is technically a privileged
guest.  The Mono runtime is analogous the JRE and there are probably games
and other situations where there is a host environment to guest programs
that don't run directly on the OS.  All should be covered.

Please forgive me, I am still confused as to what happened with CCEs.  If
CPEs are going to start describing more and more, will CCEs remain relevant?

If an executable is a dependency it is used as a library so let's just call
it a library.

Since letters are cheap and still available, how about something like "h"
for host?  The .Net runtime (or Mono) can be hosted in a browser?  This
scenario is called Siverlight or Moonlight (for the Mono inclined).  So you
have one host inside another.  It could work.

Though I can afford the 2 cents ante, I feel a little out of my depth.  Feel
free to tell me what CPEs were originally designed to describe.  Would it
make sense to have a concept as generic as a dependency and leave it at
that?

Thanks,

Vladimir Giszpenc
DSCI Contractor Supporting
US Army CERDEC S&TCD IAD Tactical Network Protection Branch
[phone]

> -----Original Message-----
> From: Ken Lassesen [mailto:[hidden email]]
> Sent: Saturday, October 06, 2007 3:37 PM
> To: [hidden email]
> Subject: [CPE-DISCUSSION-LIST] Finer resolution in CPE.
>
> My 2 cents:
>
> I am very much in favor of decomposing anything that runs on an OS (and
> some of the OS Components!) into their components (at least to support
> it!).  Many CVE's apply to many products because they all happen to use
> the same library or dll. I would suggest
> * "l" for libraries (including Controls (OCX's) other code bases that
> are not directly executed from the OS (aka .exe)
> * "j" for Java -- I feel we need to break this out into it's own
> namespace
> * "e" for executable
>
> So within the CPL (Language) a CPE for an application ("a") may be
> broken down into "l","e" and "j" parts.  A CVE may then be more
> precisely identified to the specific element of an application.
>
> As a FYI: I'm very much in favor of following Joe's suggestion of having
> CPE being a number with the CPL containing the multitude of metadata
> that we are attempting to overload the CPE ID with. Going on a crusade
> to find the holy grail is nice and noble --- but we need to put ploughs
> to the soil NOW. Put off the quest and let us get something out now that
> gives us a clean ID with the metadata being separate (and likely verbose
> in many cases).
>
>
> Ken Lassesen,
> Office 206-734-4718 Home: 360-297-4717   Cell: 360-509-2402  Skype:
> Ken.Lassesen
> IM: [hidden email]
>
> CONFIDENTIALITY NOTICE
> The information contained in this electronic message may contain
> confidential and privileged information and is intended only for use by
> the individual(s) or entity(ies) to whom it was addressed. Any
> unauthorized review, use, disclosure, or distribution of this
> communication is strictly prohibited. If you are not the intended
> recipient, please contact the sender by reply email and permanently
> delete and destroy the original message.

[Ken Lassesen-3]

A quick note (more later on reflection),when this [OVAL-SCAP] initiative
was started, virtualization existed but was so far off the beaten path
that it was not considered. Unfortunately, a generation in technology is
2-3 years, and virtualization has really grown in impact and
significance. At the NIST conference there was signs of definite pain
over it.

I suspect the virtualization may end up being a very ill fitting
bolt-on; working from the base assumption that we are working in a
virtualized environment and must think in that environment always would
yield better and prompter solutions and standards. After all 'a
non-virtualized environment is just a virtualized environemnt"....

We may all need to do a deep (and prompt) rethink of CPE and CCE in that
context. Virtualization may not be a deferable, "we'll figure it out
later" challenge to SCAP.

In some cases, some folks may need to do a crash course in
virtualization and set up virtual sandboxes to get up to speed.

Ken Lassesen,
Office 206-734-4718 Home: 360-297-4717   Cell: 360-509-2402  Skype:
Ken.Lassesen
IM: [hidden email]  

CONFIDENTIALITY NOTICE
The information contained in this electronic message may contain
confidential and privileged information and is intended only for use by
the individual(s) or entity(ies) to whom it was addressed. Any
unauthorized review, use, disclosure, or distribution of this
communication is strictly prohibited. If you are not the intended
recipient, please contact the sender by reply email and permanently
delete and destroy the original message.


-----Original Message-----
From: Vladimir Giszpenc [mailto:[hidden email]]
Sent: Tuesday, October 09, 2007 4:43 AM
To: [hidden email]
Subject: Re: [CPE-DISCUSSION-LIST] Finer resolution in CPE.

Hi Ken et al,

Granularity is always desired, but Java is not meaningful enough.  Now,
if you mean the JRE, I would counter that there are other runtime
environments.
Virtualization as a whole should be addressed.  You have it at the OS
level with VMWare, hypervisors and hardware solutions.  This creates the
need to define hosts, guests and a dom0 which (I think) is technically a
privileged guest.  The Mono runtime is analogous the JRE and there are
probably games and other situations where there is a host environment to
guest programs that don't run directly on the OS.  All should be
covered.

Please forgive me, I am still confused as to what happened with CCEs.
If CPEs are going to start describing more and more, will CCEs remain
relevant?

If an executable is a dependency it is used as a library so let's just
call it a library.

Since letters are cheap and still available, how about something like
"h"
for host?  The .Net runtime (or Mono) can be hosted in a browser?  This
scenario is called Siverlight or Moonlight (for the Mono inclined).  So
you have one host inside another.  It could work.

Though I can afford the 2 cents ante, I feel a little out of my depth.
Feel free to tell me what CPEs were originally designed to describe.
Would it make sense to have a concept as generic as a dependency and
leave it at that?

Thanks,

Vladimir Giszpenc
DSCI Contractor Supporting
US Army CERDEC S&TCD IAD Tactical Network Protection Branch
[phone]



> -----Original Message-----
> From: Ken Lassesen [mailto:[hidden email]]
> Sent: Saturday, October 06, 2007 3:37 PM
> To: [hidden email]
> Subject: [CPE-DISCUSSION-LIST] Finer resolution in CPE.
>
> My 2 cents:
>
> I am very much in favor of decomposing anything that runs on an OS
(and
> some of the OS Components!) into their components (at least to support
> it!).  Many CVE's apply to many products because they all happen to
use

> the same library or dll. I would suggest
> * "l" for libraries (including Controls (OCX's) other code bases that
> are not directly executed from the OS (aka .exe)
> * "j" for Java -- I feel we need to break this out into it's own
> namespace
> * "e" for executable
>
> So within the CPL (Language) a CPE for an application ("a") may be
> broken down into "l","e" and "j" parts.  A CVE may then be more
> precisely identified to the specific element of an application.
>
> As a FYI: I'm very much in favor of following Joe's suggestion of

having
> CPE being a number with the CPL containing the multitude of metadata
> that we are attempting to overload the CPE ID with. Going on a crusade
> to find the holy grail is nice and noble --- but we need to put
ploughs
> to the soil NOW. Put off the quest and let us get something out now
that
> gives us a clean ID with the metadata being separate (and likely
verbose

> in many cases).
>
>
> Ken Lassesen,
> Office 206-734-4718 Home: 360-297-4717   Cell: 360-509-2402  Skype:
> Ken.Lassesen
> IM: [hidden email]
>
> CONFIDENTIALITY NOTICE
> The information contained in this electronic message may contain
> confidential and privileged information and is intended only for use

by
> the individual(s) or entity(ies) to whom it was addressed. Any
> unauthorized review, use, disclosure, or distribution of this
> communication is strictly prohibited. If you are not the intended
> recipient, please contact the sender by reply email and permanently
> delete and destroy the original message.

[Ross Jekel]

Hi Drew. Thanks for the response. Answers to the questions are below. I apologize for the length, but I thought pasting in excerpts would make it easier than following links in some cases.

>Maybe a good start would be to post a few examples with the specific
>Java package and the proposed CPE Name.
>Can you elaborate on how the POM info does not follow the CPE naming
>rules?  Is this something that could be taken care of with some
>scripting?

Okay, I'll start with the basics. The minimal POM file example (taken from the pom reference page http://maven.apache.org/pom.html) is:

<project>
  <modelVersion>4.0.0</modelVersion>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>my-project</artifactId>
  <version>1.0</version>
</project>

The model version is the version number of the POM file, so it can be ignored. What we have left in most cases are a groupId (usually a domain name, or just the primary name of the organization or sometimes the major application in cases of popular packages), the artifactId which corresponds to the library or application name, and a version.

The example above would be stored at http://repo1.maven.org/maven2/org/codehaus/mojo/my-project/1.0 (if it existed).

In the original email, I mentioned the struts package, which is used in many companies to create both internal and external systems and it already has some security bulletins against it (see http://nvd.nist.gov/nvd.cfm?cvename=CVE-2006-1546).

The POM file for struts mentioned in CVE-2006-1546 starts as follows:

<project>
<modelVersion>4.0.0</modelVersion>
<groupId>struts</groupId>
<artifactId>struts</artifactId>
<name>struts</name>
<version>1.2.8</version>
<url>http://struts.apache.org/</url>
...
</project>

While in this case, we have a URL in the POM file that can be used to infer that Apache is the organization, not all POM files include the URL element. Possible CPEs for this would be:

1. cpe:/[part]:apache:struts:1.2.8
2. cpe:/[part]:struts:struts:1.2.8 (direct mapping)

I've left [part] unspecified as I think we need to decide on a Java or POM letter. #1 most closely follows the CPE specification. The trouble with it is that from the CPE one may not immediately be able to derive the POM URL which would be quite useful for gathering further meta-data about the version. Linking CPE <-> POM in both directions makes sense for POMs as they already have a unique name space, but it doesn't necessarily follow the CPE rules to the letter. #2, for example, would be a more direct mapping of groupId, artifactId, and name. So, it is the vendor component that I believe may need some relaxation of rules were there to be a direct two-way mapping of CPE <-> POM.

Another example here for Axis 1.0 (http://nvd.nist.gov/nvd.cfm?cvename=CVE-2007-2353), which has a minimal POM file (http://repo1.maven.org/maven2/axis/axis/1.0/axis-1.0.pom):

<project>
<modelVersion>4.0.0</modelVersion>
<groupId>axis</groupId>
<artifactId>axis</artifactId>
<version>1.0</version>
</project>

Again the possibilities are:

1. cpe:/[part]:apache:axis:1.0
2. cpe:/[part]:axis:axis:1.0 (direct mapping)

Note that this POM has no url item to help in the translation from POM to #1's CPE.

>But with libraries, you don't have a
>platform with the library installed, instead you might have a platform
>with an application installed that uses the library.  Subtle but
>interesting difference.  I probably just need to change the way I think
>about CPE.
>What do others think?

I would like to hear what others think here. Most of our support customers are creating custom applications on a "platform" of a certain OS, plus a "stack" of Java components that give them a website framework and transactional persistence capabilities. The finally custom "application" most likely does not need a CPE, but every component they use I think should (since two of them mentioned above already have CPEs against them).

I'd also like to make it clear that Java often isn't available from Linux vendors (in part because of the Sun License) and certainly doesn't come with Windows, and as such much of software available for Java also tends to be distributed through channels other than the OS vendors themselves. In a sense, a Java package almost needs two OS parts (the Host OS/Version and the Version of the VM), so I definitely think a strict triple of (h, o, a) parts is a bit out of date. I believe CPEs should just enumerate reality, rather than trying to be idealistic. Why shouldn't we have a unique name for each part of a system (run by major Fortune 500 companies) such as hardware, virtualization software, OS, webserver (an OS of sorts), Java or other VM, middleware components (that might not be "applications" themselves in the strictest sense). We could/should in fact perhaps even be enumerating versions of file systems (I know this could be argued to be a configuration of a given OS, but file systems can be components in their own right, no)?

What about CPE names for a particular Linux kernel number apart from a vendor. Sure you can list every vendor, but there are also people that need to match against a vanilla kernel version, right?

Anyway, I didn't mean to get off topic. The main point is there is a wealth of software for the "Java" OS that already has an available name space that could be useful in enumerating a large percentage of applications and components that might end up having bulletins against them.

As one final request, could we add a letter to the [part] that would allow for user extensions to the CPE namespace? I'm thinking maybe if a part has a prefix of 'u' (like "uh", "uo", "ua") then it is a CPE that only make sense within the user's data model. Adding such would allow me to have a uniform CPE URL syntax for all components my system needs to track without having to wait for the CPE specification to catch up.

Ross