[PATCHv2 0/3] gitweb: Smarter snapshot names
|
|
Jakub Narebski
|
This is the replacement for 'mr/gitweb-snapshot' branch (aba1076),
currently in 'pu'. Compared to previous series the t9502-gitweb-standalone-parse-output test is created in earlier commit, to better show changes in snapshot behaviour in "gitweb: Smarter snapshot names". The first commit in series is identical to the one in previous version of this series. Changes in last commit in series are only about tests. Shortlog: ~~~~~~~~~ Jakub Narebski (2): t/gitweb-lib.sh: Split gitweb output into headers and body gitweb: Document current snapshot rules via new tests Mark Rada (1): gitweb: Smarter snapshot names Table of contents: ~~~~~~~~~~~~~~~~~~ [PATCH 1/3] t/gitweb-lib.sh: Split gitweb output into headers and body [PATCH 2/3] gitweb: Document current snapshot rules via new tests [PATCH 3/3] gitweb: Smarter snapshot names Diffstat: ~~~~~~~~~ gitweb/gitweb.perl | 76 +++++++++++++++---- t/gitweb-lib.sh | 6 +- t/t9502-gitweb-standalone-parse-output.sh | 115 +++++++++++++++++++++++++++++ 3 files changed, 180 insertions(+), 17 deletions(-) create mode 100755 t/t9502-gitweb-standalone-parse-output.sh -- Jakub Narebski Poland -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to [hidden email] More majordomo info at http://vger.kernel.org/majordomo-info.html |
||||||||||||||||
|
Jakub Narebski
|
Save HTTP headers into gitweb.headers, and the body of message into
gitweb.body in gitweb_run() Signed-off-by: Jakub Narebski <[hidden email]> --- This version is identical to the one in previous version of this series. I hope that this use of sed is portable enough. Please take into account that HTTP headers part contains at least one line: HTTP status. t/gitweb-lib.sh | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diff --git a/t/gitweb-lib.sh b/t/gitweb-lib.sh index 8452532..32b841d --- a/t/gitweb-lib.sh +++ b/t/gitweb-lib.sh @@ -52,10 +52,14 @@ gitweb_run () { rm -f gitweb.log && perl -- "$SCRIPT_NAME" \ >gitweb.output 2>gitweb.log && + sed -e '/^\r$/q' <gitweb.output >gitweb.headers && + sed -e '1,/^\r$/d' <gitweb.output >gitweb.body && if grep '^[[]' gitweb.log >/dev/null 2>&1; then false; else true; fi # gitweb.log is left for debugging - # gitweb.output is used to parse http output + # gitweb.output is used to parse HTTP output + # gitweb.headers contains only HTTP headers + # gitweb.body contains body of message, without headers } . ./test-lib.sh -- 1.6.5 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to [hidden email] More majordomo info at http://vger.kernel.org/majordomo-info.html |
||||||||||||||||
|
Jakub Narebski
|
In reply to this post
by Jakub Narebski
Add t9502-gitweb-standalone-parse-output test script, which runs
gitweb as a CGI script from the commandline and checks that it produces the correct output. Currently this test script contains only tests of snapshot naming (proposed name of snapshot file) and snapshot prefix (prefix of files in the archive / snapshot). It defines and uses 'tar' snapshot format, without compression, for easy checking of snapshot prefix. Testing is done using check_snapshot function. Gitweb uses the following format for snapshot filenames: <sanitized project name>-<hash parameter><snapshot suffix> where <sanitized project name> is project name with '.git' or '/.git' suffix stripped, unless '.git' is the whole project name. For snapshot prefix it uses simply: <sanitized project name>/ Disadvantages of current snapshot rules: * There exists convention that <basename>.<suffix> archive unpacks to <basename>/ directory (<basename>/ is prefix of archive). Gitweb does not respect it * Snapshot links generated by gitweb use full SHA-1 id as a value of 'h' / $hash parameter. With current rules it leads to long file names like e.g. repo-1005c80cc11c531d327b12195027cbbb4ff9e3cb.tgz * For handcrafted URLs, where 'h' / $hash parameter is a symbolic 'volatile' revision name such as "HEAD" or "next" snapshot name doesn't tell us what exact version it was created from * Proposed filename in Content-Disposition header should not contain any directory path information, which means that it should not contain '/' (see RFC2183)... which means that snapshot naming is broken for $hash being e.g. hirearchical branch name such as 'xx/test' This would be improved in next commit. Signed-off-by: Jakub Narebski <[hidden email]> --- This is a new commit in this series. It wasn't present in previous version of "smarter snapshot names" series. t/t9502-gitweb-standalone-parse-output.sh | 87 +++++++++++++++++++++++++++++ 1 files changed, 87 insertions(+), 0 deletions(-) create mode 100755 t/t9502-gitweb-standalone-parse-output.sh diff --git a/t/t9502-gitweb-standalone-parse-output.sh b/t/t9502-gitweb-standalone-parse-output.sh new file mode 100755 index 0000000..741187b --- /dev/null +++ b/t/t9502-gitweb-standalone-parse-output.sh @@ -0,0 +1,87 @@ +#!/bin/sh +# +# Copyright (c) 2009 Mark Rada +# + +test_description='gitweb as standalone script (parsing script output). + +This test runs gitweb (git web interface) as a CGI script from the +commandline, and checks that it produces the correct output, either +in the HTTP header or the actual script output.' + + +. ./gitweb-lib.sh + +# ---------------------------------------------------------------------- +# snapshot file name and prefix + +cat >>gitweb_config.perl <<\EOF + +$known_snapshot_formats{'tar'} = { + 'display' => 'tar', + 'type' => 'application/x-tar', + 'suffix' => '.tar', + 'format' => 'tar', +}; + +$feature{'snapshot'}{'default'} = ['tar']; +EOF + +# Call check_snapshot with the arguments "<basename> [<prefix>]" +# +# This will check that gitweb HTTP header contains proposed filename +# as <basename> with '.tar' suffix added, and that generated tarfile +# (gitweb message body) has <prefix> as prefix for al files in tarfile +# +# <prefix> default to <basename> +check_snapshot () { + basename=$1 + prefix=${2:-"$1"} + echo "basename=$basename" + grep "filename=.*$basename.tar" gitweb.headers >/dev/null 2>&1 && + "$TAR" tf gitweb.body >file_list && + ! grep -v "^$prefix/" file_list +} + +test_expect_success setup ' + test_commit first foo && + git branch xx/test && + FULL_ID=$(git rev-parse --verify HEAD) && + SHORT_ID=$(git rev-parse --verify --short=7 HEAD) +' +test_debug ' + echo "FULL_ID = $FULL_ID" + echo "SHORT_ID = $SHORT_ID" +' + +test_expect_success 'snapshot: full sha1' ' + gitweb_run "p=.git;a=snapshot;h=$FULL_ID;sf=tar" && + check_snapshot ".git-$FULL_ID" ".git" +' +test_debug 'cat gitweb.headers && cat file_list' + +test_expect_success 'snapshot: shortened sha1' ' + gitweb_run "p=.git;a=snapshot;h=$SHORT_ID;sf=tar" && + check_snapshot ".git-$SHORT_ID" ".git" +' +test_debug 'cat gitweb.headers && cat file_list' + +test_expect_success 'snapshot: HEAD' ' + gitweb_run "p=.git;a=snapshot;h=HEAD;sf=tar" && + check_snapshot ".git-HEAD" ".git" +' +test_debug 'cat gitweb.headers && cat file_list' + +test_expect_success 'snapshot: short branch name (master)' ' + gitweb_run "p=.git;a=snapshot;h=master;sf=tar" && + check_snapshot ".git-master" ".git" +' +test_debug 'cat gitweb.headers && cat file_list' + +test_expect_failure 'snapshot: hierarchical branch name (xx/test)' ' + gitweb_run "p=.git;a=snapshot;h=xx/test;sf=tar" && + ! grep "filename=.*/" gitweb.headers +' +test_debug 'cat gitweb.headers' + +test_done -- 1.6.5 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to [hidden email] More majordomo info at http://vger.kernel.org/majordomo-info.html |
||||||||||||||||
|
Jakub Narebski
|
In reply to this post
by Jakub Narebski
From: Mark Rada <[hidden email]>
Teach gitweb how to produce nicer snapshot names by only using the short hash id. If clients make requests using a tree-ish that is not a partial or full SHA-1 hash, then the short hash will also be appended to whatever they asked for. If clients request snapshot of a tag (which means that $hash ('h') parameter has 'refs/tags/' prefix), use only tag name. Update tests cases in t9502-gitweb-standalone-parse-output. Gitweb uses the following format for snapshot filenames: <sanitized project name>-<version info>.<snapshot suffix> where <sanitized project name> is project name with '.git' or '/.git' suffix stripped, unless '.git' is the whole project name. For snapshot prefix it uses: <sanitized project name>-<version info>/ as compared to <sanitized project name>/ before (without version info). Current rules for <version info>: * if 'h' / $hash parameter is SHA-1 or shortened SHA-1, use SHA-1 shortened to to 7 characters * otherwise if 'h' / $hash parameter is tag name (it begins with 'refs/tags/' prefix, use tag name (with 'refs/tags/' stripped * otherwise if 'h' / $hash parameter starts with 'refs/heads/' prefix, strip this prefix, convert '/' into '.', and append shortened SHA-1 after '-', i.e. use <sanitized hash>-<shortened sha1> Signed-off-by: Mark Rada <[hidden email]> Signed-off-by: Shawn O. Pearce <[hidden email]> Signed-off-by: Jakub Narebski <[hidden email]> --- Changes since v6: - use check_snapshot function to test snapshots; therefore separate test checking archive prefix is not needed any more - fixed checking that hierarchical branch names work correctly gitweb/gitweb.perl | 76 +++++++++++++++++++++++------ t/t9502-gitweb-standalone-parse-output.sh | 38 ++++++++++++-- 2 files changed, 93 insertions(+), 21 deletions(-) diff --git a/gitweb/gitweb.perl b/gitweb/gitweb.perl index 8d4a2ae..d8dfd95 100755 --- a/gitweb/gitweb.perl +++ b/gitweb/gitweb.perl @@ -1983,16 +1983,27 @@ sub quote_command { # get HEAD ref of given project as hash sub git_get_head_hash { - my $project = shift; + return git_get_full_hash(shift, 'HEAD'); +} + +sub git_get_full_hash { + return git_get_hash(@_); +} + +sub git_get_short_hash { + return git_get_hash(@_, '--short=7'); +} + +sub git_get_hash { + my ($project, $hash, @options) = @_; my $o_git_dir = $git_dir; my $retval = undef; $git_dir = "$projectroot/$project"; - if (open my $fd, "-|", git_cmd(), "rev-parse", "--verify", "HEAD") { - my $head = <$fd>; + if (open my $fd, '-|', git_cmd(), 'rev-parse', + '--verify', '-q', @options, $hash) { + $retval = <$fd>; + chomp $retval if defined $retval; close $fd; - if (defined $head && $head =~ /^([0-9a-fA-F]{40})$/) { - $retval = $1; - } } if (defined $o_git_dir) { $git_dir = $o_git_dir; @@ -5179,6 +5190,43 @@ sub git_tree { git_footer_html(); } +sub snapshot_name { + my ($project, $hash) = @_; + + # path/to/project.git -> project + # path/to/project/.git -> project + my $name = to_utf8($project); + $name =~ s,([^/])/*\.git$,$1,; + $name = basename($name); + # sanitize name + $name =~ s/[[:cntrl:]]/?/g; + + my $ver = $hash; + if ($hash =~ /^[0-9a-fA-F]+$/) { + # shorten SHA-1 hash + my $full_hash = git_get_full_hash($project, $hash); + if ($full_hash =~ /^$hash/ && length($hash) > 7) { + $ver = git_get_short_hash($project, $hash); + } + } elsif ($hash =~ m!^refs/tags/(.*)$!) { + # tags don't need shortened SHA-1 hash + $ver = $1; + } else { + # branches and other need shortened SHA-1 hash + if ($hash =~ m!^refs/(?:heads|remotes)/(.*)$!) { + $ver = $1; + } + $ver .= '-' . git_get_short_hash($project, $hash); + } + # in case of hierarchical branch names + $ver =~ s!/!.!g; + + # name = project-version_string + $name = "$name-$ver"; + + return wantarray ? ($name, $name) : $name; +} + sub git_snapshot { my $format = $input_params{'snapshot_format'}; if (!@snapshot_fmts) { @@ -5203,24 +5251,20 @@ sub git_snapshot { die_error(400, 'Object is not a tree-ish'); } - my $name = $project; - $name =~ s,([^/])/*\.git$,$1,; - $name = basename($name); - my $filename = to_utf8($name); - $name =~ s/\047/\047\\\047\047/g; - my $cmd; - $filename .= "-$hash$known_snapshot_formats{$format}{'suffix'}"; - $cmd = quote_command( + my ($name, $prefix) = snapshot_name($project, $hash); + my $filename = "$name$known_snapshot_formats{$format}{'suffix'}"; + my $cmd = quote_command( git_cmd(), 'archive', "--format=$known_snapshot_formats{$format}{'format'}", - "--prefix=$name/", $hash); + "--prefix=$prefix/", $hash); if (exists $known_snapshot_formats{$format}{'compressor'}) { $cmd .= ' | ' . quote_command(@{$known_snapshot_formats{$format}{'compressor'}}); } + $filename =~ s/(["\\])/\\$1/g; print $cgi->header( -type => $known_snapshot_formats{$format}{'type'}, - -content_disposition => 'inline; filename="' . "$filename" . '"', + -content_disposition => 'inline; filename="' . $filename . '"', -status => '200 OK'); open my $fd, "-|", $cmd diff --git a/t/t9502-gitweb-standalone-parse-output.sh b/t/t9502-gitweb-standalone-parse-output.sh index 741187b..dd83890 100755 --- a/t/t9502-gitweb-standalone-parse-output.sh +++ b/t/t9502-gitweb-standalone-parse-output.sh @@ -56,29 +56,57 @@ test_debug ' test_expect_success 'snapshot: full sha1' ' gitweb_run "p=.git;a=snapshot;h=$FULL_ID;sf=tar" && - check_snapshot ".git-$FULL_ID" ".git" + check_snapshot ".git-$SHORT_ID" ' test_debug 'cat gitweb.headers && cat file_list' test_expect_success 'snapshot: shortened sha1' ' gitweb_run "p=.git;a=snapshot;h=$SHORT_ID;sf=tar" && - check_snapshot ".git-$SHORT_ID" ".git" + check_snapshot ".git-$SHORT_ID" +' +test_debug 'cat gitweb.headers && cat file_list' + +test_expect_success 'snapshot: almost full sha1' ' + ID=$(git rev-parse --short=30 HEAD) && + gitweb_run "p=.git;a=snapshot;h=$ID;sf=tar" && + check_snapshot ".git-$SHORT_ID" ' test_debug 'cat gitweb.headers && cat file_list' test_expect_success 'snapshot: HEAD' ' gitweb_run "p=.git;a=snapshot;h=HEAD;sf=tar" && - check_snapshot ".git-HEAD" ".git" + check_snapshot ".git-HEAD-$SHORT_ID" ' test_debug 'cat gitweb.headers && cat file_list' test_expect_success 'snapshot: short branch name (master)' ' gitweb_run "p=.git;a=snapshot;h=master;sf=tar" && - check_snapshot ".git-master" ".git" + ID=$(git rev-parse --verify --short=7 master) && + check_snapshot ".git-master-$ID" +' +test_debug 'cat gitweb.headers && cat file_list' + +test_expect_success 'snapshot: short tag name (first)' ' + gitweb_run "p=.git;a=snapshot;h=first;sf=tar" && + ID=$(git rev-parse --verify --short=7 first) && + check_snapshot ".git-first-$ID" +' +test_debug 'cat gitweb.headers && cat file_list' + +test_expect_success 'snapshot: full branch name (refs/heads/master)' ' + gitweb_run "p=.git;a=snapshot;h=refs/heads/master;sf=tar" && + ID=$(git rev-parse --verify --short=7 master) && + check_snapshot ".git-master-$ID" +' +test_debug 'cat gitweb.headers && cat file_list' + +test_expect_success 'snapshot: full tag name (refs/tags/first)' ' + gitweb_run "p=.git;a=snapshot;h=refs/tags/first;sf=tar" && + check_snapshot ".git-first" ' test_debug 'cat gitweb.headers && cat file_list' -test_expect_failure 'snapshot: hierarchical branch name (xx/test)' ' +test_expect_success 'snapshot: hierarchical branch name (xx/test)' ' gitweb_run "p=.git;a=snapshot;h=xx/test;sf=tar" && ! grep "filename=.*/" gitweb.headers ' -- 1.6.5 -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to [hidden email] More majordomo info at http://vger.kernel.org/majordomo-info.html |
||||||||||||||||
| Free Embeddable Forum Powered by Nabble | Help |
