farkuhar/pkg-get
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

pkg-repgen: do more tasks in parallel

Browse Source 
update the html index too, when individual packages are given

pkg-get docs: explain in more detail the effect of -r (alternate root)
This commit is contained in:
John McQuah 2023-06-18 18:25:55 -04:00
parent f769b5251e
commit 57f3b29e2e
8 changed files with 469 additions and 383 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

11
ChangeLog
View File

@ -1,9 +1,14 @@
ChangeLog for pkg-get
0.4.8 - Fix pre- and post-install scripts to accommodate rootfs other
than "/", and ports with dashes in their names
- Use Perl modules where possible, to avoid spawning external processes
0.4.8 - Fix doscript() to accommodate rootfs other than "/",
and ports with dashes in their names
- Consolidate code
- Do more tasks in parallel when generating the metadata
- Update the html index too, when individual packages are passed to
pkg-repgen
- Use native Perl routines to reduce the number of external processes
- Replace wget by curl, to avoid depending on packages outside 'core'
- Update man-pages and README
0.4.7 - Fix man-page location

28
README
View File

@ -51,9 +51,17 @@ LIMITATIONS
The client and the server must be configured to use the same
pkgmk compression mode, otherwise the client will try to download
a tarball with the wrong suffix. This is only a problem if you sometimes
compile ports on the client device. By allowing you to maintain your
client device solely with binary packages, pkg-get makes the contents of
/etc/pkgmk.conf mostly irrelevant.
compile ports on the client machine and prefer a different compression mode
(better suited to its less-powerful hardware?). By allowing you to maintain
your client machine solely with binary packages, pkg-get makes the contents
of /etc/pkgmk.conf mostly irrelevant, so you can simply put a verbatim copy
of the server's pkgmk.conf on the client machine.
The pkg-get configuration file does not offer as many settings as the one
for prt-get. In particular, you cannot change "addcommand", "rmcommand", or
"runscriptscommand"; these are hard-coded as /usr/bin/pkgadd, /usr/bin/pkgrm,
and /bin/bash, respectively. There is also currently no mechanism to respect
"preferhigher", when doing 'pkg-get diff' or 'pkg-get sysup'.
'pkg-get depends' and 'prt-get quickdep' do not handle more than one port,
unlike the corresponding commands in prt-get. Therefore it is not as
@ -65,12 +73,14 @@ Alas, such a switch is also absent from the design of pkg-get. Use
the --test switch with prt-get itself, for the closest approximation
of previewing the outcome from a 'pkg-get depinst' operation.
Among the prt-get commands that have no counterpart in pkg-get
(grpinst, fsearch, deptree, listorphans, ls, cat, edit, cache),
only the 'grpinst' command is of possible interest; the remaining
commands are just as easily delegated to prt-get itself. If you want
a Perl implementation that does provide these missing commands, consider
the script written by user farkuhar [1].
'pkg-get dependent' does not support the --recursive option. Other
useful prt-get commands (grpinst, fsearch, deptree, listorphans, ls,
cat, edit, cache) have no counterpart in pkg-get. Of these omissions,
only the 'grpinst' command is of possible interest for binary package
management; the unimplemented commands and options are just as easily
delegated to prt-get itself. If you want a Perl implementation that does
provide these missing commands, consider the script written by user
farkuhar [1].
pkg-get only makes use of the hard dependencies listed by the port
maintainer, not any of the eager linking that might have occurred on the

74
doc/pkg-get.8
View File

@ -1,4 +1,3 @@
." Text automatically generated by txt2man-1.4.7
.TH pkg-get 8 "July 13, 2006" "" ""
.SH NAME
\fBpkg-get \fP- a package management tool for CRUX Linux
@ -20,30 +19,30 @@ can be generated by repository maintainers with the
.TP
.B
install <package1> [<package2>..<packageN>]
Install given packages,
download if necessary.
Install given packages, download if necessary.
.TP
.B
update <package1> [<package2>..<packageN>]
Update given packages,
download if necessary.
Update given packages, download if necessary.
.TP
.B
diff [\fB--all\fP]
Show a list of outdated packages. The \fB--all\fP option also displays locked packages.
Show a list of outdated packages.
The \fB--all\fP option also displays locked packages.
.TP
.B
quickdiff
Show a compact list of outdated packages.
.TP
.B
sysup
Update all outdated packages. Download if necessary.
sysup [\fB--all\fP]
Update all outdated packages, download if necessary.
Locked packages are excluded from the operation
unless \fB--all\fP is passed.
.TP
.B
depinst <package1> [<package2>..<packageN>]
Install given packages
and relative dependencies.
Install given packages and relative dependencies.
.TP
.B
depends <package>
@ -51,13 +50,17 @@ Show a recursive list of dependencies for package
and their installation status.
.TP
.B
quickdep <package>
Show a brief list of dependencies for package.
.TP
.B
dependent <package> [\fB--all\fP]
Show installed (or all with the \fB--all\fP option) packages
that depend from package.
.TP
.B
quickdep <package>
Show a brief list of dependencies for package.
path <package>
Show local path of package.
.TP
.B
info <package>
@ -68,8 +71,8 @@ current <package>
Show currently installed version of package.
.TP
.B
path <package>
Show local path of package.
isinst <package>
Display whether a package is installed.
.TP
.B
readme <package>
@ -84,10 +87,6 @@ listinst
List all installed packages.
.TP
.B
isinst <package>
Display whether a package is installed.
.TP
.B
dsearch <string>
Search for packages which name or description contain <string>.
.TP
@ -101,9 +100,9 @@ List all duplicates ports (present in more than one repository).
.TP
.B
printf <format string1> [\fB--filter\fP=<filter>]
Description blatantly
stolen from prt-get man file. Print formatted port list. Format string can
contain variables, which are replaced like this:
Description blatantly stolen from \fBprt\-get\fP(8).
Print formatted port list. Format string can contain variables,
which are replaced like this:
.RS
.IP \(bu 3
%n -> name
@ -128,7 +127,7 @@ contain variables, which are replaced like this:
.IP \(bu 3
%M -> "Nobody". for compatibility with prt-get
.IP \(bu 3
%P -> "Nobody". for compatibility with prt-get
%P -> "None". for compatibility with prt-get
.IP \(bu 3
%l -> is locked ("yes"/"no")
.IP \(bu 3
@ -138,7 +137,7 @@ repository.
.RE
.PP
Use "\\n" and "\\t" to format your output (no additional format
specified suported). The optional format string2 can contain the
codes suported). The optional format string2 can contain the
same variables as format string1 and is used to sort the output.
You can specify a wildcard filter to filter by package name.
.TP
@ -160,12 +159,8 @@ Syncronize local packages with the ones from the remote repository.
If the repository is local, this \fIcommand\fP does nothing.
.TP
.B
sysup
Update all outdated packages.
.TP
.B
help
Display brief help screen.
Display brief help screen.
.TP
.B
version
@ -174,12 +169,15 @@ Show \fBpkg-get\fP version.
.TP
.B
\fB-r\fP <root>
Use <root> directory when wrapping pkgadd. Note that
this only works with update / install !
Use <root> directory for operations involving the package database
(install, depinst, update, isinst, current, listinst, diff, quickdiff, sysup).
This option does not affect the directories from which \fBpkg-get\fP loads the
metadata or tarballs; those settings are still governed by the entries in
the configuration file.
.TP
.B
\fB-do\fP
Download only. Applicable to: sysup, depinst, install, update.
Download only. Applicable to: sysup, install, depinst, update.
.TP
.B
\fB-f\fP
@ -199,15 +197,15 @@ Use the specified configuration file.
.TP
.B
\fB--pre-install\fP
executes pre-install script if available.
Execute pre-install script if available.
.TP
.B
\fB--post-install\fP
executes post-install script if available.
Execute post-install script if available.
.TP
.B
\fB--install-scripts\fP
executes pre-install and post-install
Execute pre-install and post-install
scripts if available.
.RE
.PP
@ -222,13 +220,13 @@ Configuration is handled by the /etc/pkg-get.conf file,
Install sqlite and pysqlite.
.TP
.B
\fBpkg-get\fP depinst kdebase \fB-f\fP
Install kdebase and all its dependencies, forcing upgrade.
\fBpkg-get\fP depinst qt6-base \fB-f\fP
Install qt6-base and all its dependencies, forcing upgrade.
.TP
.B
\fBpkg-get\fP sysup \fB-do\fP
Download new releases of all the outdated packages.
.SH AUTHORS
Simone Rota <sip@varlock.com>
Simone Rota <sip@varlock.com>, John McQuah <jmcquah@disroot.org>
.SH SEE ALSO
\fBpkgadd\fP(8), \fBprt-get\fP(8)
\fBpkgadd\fP(8), \fBprt-get\fP(8), \fBpkg-repgen\fP(8)

245
doc/pkg-get.8.txt
View File

@ -1,101 +1,176 @@
pkg-get(8)
NAME
pkg-get - a package management tool for CRUX Linux
pkg-get - a package management tool for CRUX Linux
SYNOPSIS
pkg-get command <arguments> [options]
pkg-get command <arguments> [options]
DESCRIPTION
pkg-get is a simple package management tool for CRUX Linux.
It tries to replicate some of the most useful features of the
port management tool prt-get(8) to be used with binary packages.
pkg-get requires a remote or local package repository that
can be generated by repository maintainers with the
pkg-repgen(8) script.
pkg-get is a simple package management tool for CRUX Linux.
It tries to replicate some of the most useful features of the
port management tool prt-get(8) to be used with binary packages.
pkg-get requires a remote or local package repository that can be
generated by repository maintainers with the pkg-repgen(8) script.
COMMANDS
install <package1> [<package2>..<packageN>] Install given packages,
download if necessary.
update <package1> [<package2>..<packageN>] Update given packages,
download if necessary.
diff [--all] Show a list of outdated packages. The --all option also displays locked packages.
quickdiff Show a compact list of outdated packages.
sysup Update all outdated packages. Download if necessary.
depinst <package1> [<package2>..<packageN>] Install given packages
and relative dependencies.
depends <package> Show a recursive list of dependencies for package
and their installation status.
dependent <package> [--all] Show installed (or all with the --all option) packages
that depend from package.
quickdep <package> Show a brief list of dependencies for package.
info <package> Show information about package.
current <package> Show currently installed version of package.
path <package> Show local path of package.
readme <package> Print README information (if available) for the package.
list List all packages in the repository.
listinst List all installed packages.
isinst <package> Display whether a package is installed.
dsearch <string> Search for packages which name or description contain <string>.
search <string> Search for packages which name contains <string>.
dup List all duplicates ports (present in more than one repository).
printf <format string1> [--filter=<filter>] Description blatantly
stolen from prt-get man file. Print formatted port list. Format string can
contain variables, which are replaced like this:
- %n -> name
- %p -> path
- %v -> version
- %r -> release
- %d -> description
- %e -> dependencies
- %u -> url
- %R -> Readme ("yes"/"no")
- %E -> pre-install script ("yes"/"no")
- %O -> post-install script ("yes"/"no")
- %M -> "Nobody". for compatibility with prt-get
- %P -> "Nobody". for compatibility with prt-get
- %l -> is locked ("yes"/"no")
- %i -> "no" if not installed, "yes" if it's installed and up to
date and "diff" if it's installed and a new version is in the
repository.
Use "\\n" and "\\t" to format your output (no additional format
specified suported). The optional format string2 can contain the
same variables as format string1 and is used to sort the output.
You can specify a wildcard filter to filter by package name.
install <package1> [<package2>..<packageN>]
Install given packages, download if necessary.
update <package1> [<package2>..<packageN>]
Update given packages, download if necessary.
diff [--all]
Show a list of outdated packages. The --all option also
displays locked packages.
quickdiff
Show a compact list of outdated packages.
sysup [--all]
Update all outdated packages, download if necessary.
Locked packages are excluded from the operation unless
--all is passed.
depinst <package1> [<package2>..<packageN>]
Install given packages and relative dependencies.
depends <package>
Show a recursive list of dependencies for package and
their installation status.
quickdep <package>
Show a brief list of dependencies for package.
dependent <package> [--all]
Show installed (or all with the --all option) packages
that depend from package.
path <package>
Show local path of package.
info <package>
Show information about package.
current <package>
Show currently installed version of package.
isinst <package>
Display whether a package is installed.
readme <package>
Print README information (if available) for the package.
list List all packages in the repository.
listinst
List all installed packages.
dsearch <string>
Search for packages whose name or description contains
<string>.
search <string>
Search for packages whose name contains <string>.
dup List all duplicate ports (present in more than one
repository).
printf <format string1> [--filter=<filter>]
Description blatantly stolen from prt-get(8). Print
formatted port list. Format string can contain variables,
which are replaced like this:
• %n -> name
• %p -> path
• %v -> version
• %r -> release
• %d -> description
• %e -> dependencies
• %u -> url
• %R -> Readme ("yes"/"no")
• %E -> pre-install script ("yes"/"no")
• %O -> post-install script ("yes"/"no")
• %M -> "Nobody". for compatibility with prt-get
• %P -> "None". for compatibility with prt-get
• %l -> is locked ("yes"/"no")
• %i -> "no" if not installed, "yes" if it's installed
and up to date and "diff" if it's installed and
a new version is in the repository.
Use "\n" and "\t" to format your output (no additional format
codes supported). The optional format string2 can contain
the same variables as format string1 and is used to sort
the output. You can specify a wildcard filter to filter by
package name.
lock <package1> [<package2>..<packageN>]
Lock a package (ignore updates).
unlock <package1> [<package2>..<packageN>]
Unlock a package.
listlocked
Display a list of locked packages.
sync Syncronize local packages with the ones from the remote
repository. If the repository is local, this command does
nothing.
help Display brief help screen.
version
Show pkg-get version.
lock <package1> [<package2>..<packageN>] Lock a package (ignore updates).
unlock <package1> [<package2>..<packageN>] Unlock a package.
listlocked Display a list of locked packages.
sync Syncronize local packages with the ones from the remote repository.
If the repository is local, this command does nothing.
sysup Update all outdated packages.
help Display brief help screen.
version Show pkg-get version.
OPTIONS
-r <root> Use <root> directory when wrapping pkgadd. Note that
this only works with update / install !
-do Download only. Applicable to: sysup, depinst, install, update.
-f Force installing / upgrading. This is passed to pkgadd(8).
-im Ignore md5sum mismatches.
--aargs="arguments" pass the specified arguments to pkgadd(8).
--config=/path/to/file Use the specified configuration file.
--pre-install executes pre-install script if available.
--post-install executes post-install script if available.
--install-scripts executes pre-install and post-install
scripts if available.
-r <root>
Use <root> directory for operations involving the package
database (install, depinst, update, isinst, current,
listinst, diff, quick‐ diff, sysup). This option does
not affect the directories from which pkg-get loads the
metadata or tarballs; those settings are still governed
by the entries in the configuration file.
-do Download only. Applicable to: sysup, install, depinst,
update.
-f Force installing / upgrading. This is passed to pkgadd(8).
-im Ignore md5sum mismatches.
--aargs="arguments"
Pass the specified arguments to pkgadd(8).
--config=/path/to/file
Use the specified configuration file.
--pre-install
Execute pre-install script if available.
--post-install
Execute post-install script if available.
--install-scripts
Execute pre-install and post-install scripts if available.
CONFIGURATION
Configuration is handled by the /etc/pkg-get.conf file,
options are explained in the file itself.
Configuration is handled by the /etc/pkg-get.conf file, options
are explained in the file itself.
EXAMPLES
pkg-get install sqlite pysqlite Install sqlite and pysqlite.
pkg-get depinst kdebase -f Install kdebase and all its dependencies, forcing upgrade.
pkg-get sysup -do Download new releases of all the outdated packages.
pkg-get install sqlite pysqlite
Install sqlite and pysqlite.
pkg-get depinst qt6-base -f
Install qt6-base and all its dependencies, forcing upgrade.
pkg-get sysup -do
Download new releases of all the outdated packages.
AUTHORS
Simone Rota <sip@varlock.com>
Simone Rota <sip@varlock.com>, John McQuah <jmcquah@disroot.org>
SEE ALSO
pkgadd(8), prt-get(8)
pkgadd(8), prt-get(8), pkg-repgen(8)
July 13, 2006 pkg-get(8)

14
doc/pkg-repgen.8
View File

@ -1,4 +1,3 @@
." Text automatically generated by txt2man-1.4.7
.TH pkg-repgen 8 "July 13, 2006" "" ""
.SH NAME
\fBpkg-repgen \fP- generate a package repository for pkg-get
@ -13,12 +12,13 @@
\fBpkg-repgen\fP generates files needed by a \fBpkg-get\fP(8) package
repository. It also generates a html index of the packages.
.SH USAGE
run \fBpkg-repgen\fP from the directory containing packages to
Run \fBpkg-repgen\fP from the directory containing packages to
generate the whole repository.
Specify a list of package names to update only those packages
(this is expecially useful with large repositories).
Note that the html index is not updated when single
packages are specified.
(this is especially useful with large repositories).
Even when individual packages are specified, the entire directory
is still processed to collect all the readmes and {pre,post}-install
scripts.
.SH OPTIONS
.TP
.B
@ -39,9 +39,9 @@ use alternative prtdir for prt-get commands
.SH EXAMPLES
\fBpkg-repgen\fP
.PP
\fBpkg-repgen\fP kdebase kdelibs
\fBpkg-repgen\fP qt6-base qt6-tools
.SH AUTHORS
Simone Rota <sip@varlock.com>
Simone Rota <sip@varlock.com>, John McQuah <jmcquah@disroot.org>
index generation code adapted from Jukka Heino's portspage
.SH SEE ALSO
\fBpkg-get\fP(8), \fBprt-get\fP(8)

51
doc/pkg-repgen.8.txt
View File

@ -1,35 +1,46 @@
pkg-repgen(8)
NAME
pkg-repgen - generate a package repository for pkg-get
pkg-repgen - generate a package repository for pkg-get
SYNOPSIS
pkg-repgen [options][package1...packageN]
pkg-repgen [options][package1...packageN]
DESCRIPTION
pkg-repgen generates files needed by a pkg-get(8) package
repository. It also generates a html index of the packages.
pkg-repgen generates files needed by a pkg-get(8) package
repository. It also generates a html index of the packages.
USAGE
run pkg-repgen from the directory containing packages to
generate the whole repository.
Specify a list of package names to update only those packages
(this is expecially useful with large repositories).
Note that the html index is not updated when single
packages are specified.
Run pkg-repgen from the directory containing packages to generate
the whole repository. Specify a list of package names to
update only those packages (this is especially useful with large
repositories). Even when individual packages are specified,
the entire directory is still processed to collect all the
readmes and {pre,post}-install scripts.
OPTIONS
--header=FILE insert FILE at the beginning of the html index
--header=FILE insert FILE at the beginning of the html index
--title=title use the specified title for the index page
--prtdir=DIR use alternative prtdir for prt-get commands
--header=FILE
insert FILE at the beginning of the html index
--header=FILE
insert FILE at the beginning of the html index
--title=title
use the specified title for the index page
--prtdir=DIR
use alternative prtdir for prt-get commands
EXAMPLES
pkg-repgen
pkg-repgen kdebase kdelibs
pkg-repgen
pkg-repgen qt6-base qt6-tools
AUTHORS
Simone Rota <sip@varlock.com>
index generation code adapted from Jukka Heino's portspage
Simone Rota <sip@varlock.com>, John McQuah <jmcquah@disroot.org>
index generation code adapted from Jukka Heino's portspage
SEE ALSO
pkg-get(8), prt-get(8)
pkg-get(8), prt-get(8)
July 13, 2006 pkg-repgen(8)

66
scripts/pkg-get.pl
View File

@ -2,7 +2,7 @@
#
# pkg-get - A binary package management utility for CRUX Linux
# Copyright (C) 2004-2006 Simone Rota <sip@varlock.com>
# Copyright (c) 2006-2013 by CRUX team (http://crux.nu)
# Copyright (C) 2006-2023 by CRUX team (http://crux.nu)
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@ -79,7 +79,7 @@ SWITCH: {
if ($command eq "listlocked") { listlocked(); last SWITCH; }
if ($command eq "printf") { doprintf(); last SWITCH; }
if ($command =~ /^(isinst|current)$/) { current(); last SWITCH; }
if ($command =~ /^(|quick)diff$/) { diff($1); last SWITCH; }
if ($command =~ /^(diff|quickdiff|sysup)$/) { diff($1); last SWITCH; }
if ($command eq "dup") { dup(); last SWITCH; }
if ($command =~ /^(depends|quickdep)$/) { depends(); last SWITCH; }
if ($command =~ /^(install|update)$/) { upinst(@ARGV); last SWITCH; }
@ -151,7 +151,6 @@ sub readconfig {
# Populate a hash of locked packages
sub get_locked {
(-f $LOCKFILE) or system("touch $LOCKFILE");
open (my $fL, $LOCKFILE) or return;
while (<$fL>) { $locked{$_} = 1; }
close ($fL);
@ -439,7 +438,12 @@ sub installpkg {
$pptasks{$pkg{'name'}} = "";
if ($download_only) {return 1;}
if ($force){$aa = $aa."-f ";}
if ($root) {$aa = $aa."-r ".$root." ";}
if ($root) {
$aa = $aa."-r ".$root." ";
(-f "$root/$pkg{'path'}/PKGINST") or
system("install -D $pkg{'path'}/PKGINST $root/$pkg{'path'}/PKGINST")
or die "Failed to copy PKGINST to $root, aborting.\n";
}
if ($pkg{'pre_install'} eq "yes" and ($install_scripts or $pre_install)) {doscript("pre",%pkg);}
my $fullpath = $pkg{'path'}."/".$pkg{'name'}."#".$pkg{'version'}."-".$pkg{'release'}.".pkg.tar.$compress";
print "pkg-get: /usr/bin/pkgadd $upgrade $aa$fullpath\n";
@ -452,7 +456,7 @@ sub installpkg {
sub doscript {
my ($when, %pkg) = @_;
my $cmd = ($root) ? "chroot $root " : "";
$cmd .= "/bin/bash $pkg{'path'}/PKGINST $pkg{name} $when";
$cmd .= "/bin/bash $pkg{'path'}/PKGINST $pkg{'name'} $when";
if (system($cmd) == 0) {
$pptasks{$pkg{'name'}} .= " [$when: ok]";
} else {
@ -519,7 +523,7 @@ sub sync {
}
}
# Show info/path/readme for the package #####################################
# Show info/path/readme for the package ####################################
sub info {
my ($type, $arg) = @ARGV;
foreach my $repo(@repos) {
@ -541,7 +545,7 @@ sub info {
print "Package '$arg' not found\n";
}
# List packages containing given string (name/description) ###################
# List packages containing given string (name/description) #################
sub search {
my $arg = $ARGV[1];
my $type = ($ARGV[0] =~ /^d/) ? "desc" : "name";
@ -701,9 +705,9 @@ sub doprintf {
}
}
# Show differences between installed and available packages ################
# Show or resolve differences between installed and available packages #####
sub diff {
my $format = shift; my $gotdiff = 0; my %found;
my $format = shift; my %found; my @diff;
foreach my $repo(@repos) {
my ($dir, $url) = split(/\|/, $repo);
@ -715,20 +719,23 @@ sub diff {
$found{$pkg{'name'}} = 1;
if ($pkg{'instversion'} ne $pkg{'version'}."-".$pkg{'release'}) {
next if (($locked{$pkg{'name'}}) and (! $all));
if (($gotdiff == 0) and ($format ne "quick")) {
if (($#diff < 0) and ($format !~ /^(quick|sysup)/)) {
print "Differences between installed packages and packages repo:\n\n";
printf("%-19s %-19s %-19s\n\n","Package","Installed","Available in the repositories");
}
$gotdiff = 1;
my $lastcol = ($locked{$pkg{'name'}}) ? "locked" : $pkg{'version'}."-".$pkg{'release'};
print "$pkg{'name'} " if ($format eq "quick");
printf("%-19s %-19s %-19s\n", $pkg{'name'}, $pkg{'instversion'}, $lastcol) if ($format ne "quick");
push @diff, $pkg{'name'};
print "$pkg{'name'} " if ($format =~ /^quick/);
printf("%-19s %-19s %-19s\n", $pkg{'name'}, $pkg{'instversion'}, $lastcol) if ($format !~ /^(quick|sysup)/);
}
}
close(REPO);
}
print "\n" if (($format eq "quick") and ($gotdiff == 1));
print "No differences found\n" if ($gotdiff == 0);
print "\n" if (($format =~ /^quick/) and ($#diff >= 0));
print "No differences found\n" if ($#diff < 0);
# proceed with updates if sysup was requested
upinst("update",@diff) if (($#diff >= 0) and ($format eq "sysup"));
}
# Display duplicate packages (found in more than one repo) #################
@ -801,13 +808,8 @@ sub upinst {
my ($cmd, @args) = @_; my $aa;
($curraction, $aa) = ($cmd =~ /^up/) ? ("updated","-u") : ("installed","");
getinstalled() if (! %installed);
if ($root) {
foreach my $repo (@repos) {
my ($dir, $url) = split(/\|/, $repo);
(! -f "$dir/PKGINST") or system("install -D $dir/PKGINST $root$dir/PKGINST");
}
}
foreach my $pkgname(@args) {
my %pkg = getpackage($pkgname, 1);
if (not %pkg) {
@ -824,27 +826,7 @@ sub upinst {
printresults();
}
# Update out of date packages ##############################################
sub sysup {
my %found; my @diff;
foreach my $repo(@repos) {
my ($dir, $url) = split(/\|/, $repo);
open(REPO, "$dir/PKGREPO") or exiterr("could not open $dir/PKGREPO");
while (<REPO>) {
chomp;
my %pkg = parsepackage($_, $dir, $url, 1);
next if ($found{$pkg{'name'}}); $found{$pkg{'name'}} = 1;
if ( ($pkg{'instversion'}) and
($pkg{'instversion'} ne $pkg{'version'}."-".$pkg{'release'})
and (! $locked{$pkg{'name'}}) ) {
push(@diff, $pkg{'name'});
}
}
close(REPO);
}
upinst("update",@diff) if (@diff);
}
# Install given package, along with dependencies ###########################
sub depinst {
my @toinst; my %seen;
$curraction = "installed";

363
scripts/pkg-repgen.pl
View File

@ -22,7 +22,7 @@ GetOptions("prtdir=s"=>\$prtdir, "title=s"=>\$title, "header=s"=>\$header, "foot
our $compress = "gz";
open CONFIG, "/etc/pkgmk.conf" or die "Could not open /etc/pkgmk.conf";
while (<CONFIG>) {
$compress = $1 if m/^PKGMK_COMPRESSION_MODE=(.*)(#|$)/;
$compress = $1 if m/^PKGMK_COMPRESSION_MODE=(.*)(\#|$)/;
}
close CONFIG;
$compress =~ s/["' ]//g;
@ -32,213 +32,215 @@ $prtget .= " --no-std-config --config-set=\"prtdir $prtdir\"" if ($prtdir);
my @dirlist = glob("*#*.pkg.tar.$compress"); my @packages;
if ($#ARGV >= 0) { # single packages
foreach my $pkgname (@ARGV) {
my @hits = grep /^$pkgname#/ @dirlist;
push(@packages,@hits) if (@hits);
my @hits = sort grep { /^$pkgname\#/ } @dirlist;
push(@packages,$hits[-1]) if (@hits);
}
} else {
@packages = @dirlist;
}
# Populate some hashes with a single run of prt-get
# Populate hashes using a single run of prt-get
our %path; our %depends; our %descrip; our %flags;
our %oldDeps; our %oldFlags; our %du; our %md5sums;
fill_hashes_from_prtget();
if ($#ARGV >= 0) {
pkg_single("REPO"); pkg_single("DEPS");
} else {
pkg_dir("REPO"); pkg_dir("DEPS");
my @validkeys = @dirlist;
map { s/\#.*// } @validkeys;
my %printme = map { $_ => 1 } @validkeys;
open (my $ppf, "$prtget printf '%n^%p^%e^%d^%E:%O:%R\n' |");
while (<$ppf>) {
chomp;
my ($name,$repo,$deps,$desc,$prepostread) = split /\^/;
next if (! $printme{$name});
$path{$name} = $repo . "/" . $name;
$depends{$name} = $deps;
$desc =~ s/\:/ /g;
$descrip{$name} = $desc;
$flags{$name} = $prepostread;
}
close ($ppf);
# Needed for alternating colors in the html index
my %parity = ( 0 => "even", 1 => "odd" );
# Generate the metadata files
($#ARGV >= 0) ? pkg_single() : pkg_dir();
# Generate README and PKGINST
pkgread();
pkginst();
pkgreadscripts();
sub fill_hashes_from_prtget {
my @validkeys = @packages;
map { s/#.*// } @validkeys;
open (my $ppf, "$prtget printf '%n^%p^%e^%d^%E^%O^%R\n' |");
while (<$ppf>) {
my ($name,$repo,$deps,$desc,$haspre,$haspost,$hasreadme) = split /\^/;
next if (! grep { ($_ eq $name) } @validkeys);
$path{$name} = $repo . "/" . $name;
$depends{$name} = $deps;
$desc =~ s/\:/ /g;
$descrip{$name} = $desc;
chomp($hasreadme);
$flags{$name} = join(":", $haspre, $haspost, $hasreadme);
}
close ($ppf);
}
sub fill_hashes_from_prevrun {
my $oldRepo = shift;
open (my $fh, $oldRepo) or return;
while (<$fh>) {
chomp;
if ($oldRepo eq "PKGDEPS") {
my ($iPkg, $iDep) = split /:/;
$iPkg =~ s/\s+//g;
$iDep =~ s/,/ /g;
$oldDeps{$iPkg} = $iDep;
} elsif ($oldRepo eq "PKGREPO") {
my ($iPkg, $iSize, $iMD5, $iDesc, $iPre, $iPost, $iReadme) = split /:/;
$iPkg =~ s/\s+//g;
$oldFlags{$iPkg} = join (":", $iPre, $iPost, $iReadme);
}
}
close ($fh);
}
# generate dependency map or repository for individual packages
######################## individual packages ########################
sub pkg_single {
my $db = shift; my $name;
my $status = "+ Generating ";
$status .= ($db eq "REPO") ? "repository\n" : "dependencies\n";
print $status;
fill_hashes_from_prevrun("PKG$db");
my $hasnew = 0;
foreach my $p (@ARGV) {
my @matches = grep /^$p#/, @packages;
if ($#matches != 0) {
print "Package '$p' not found or duplicate\n"; next;
}
my $match = $matches[0];
$name = $match;
$name =~ s/#.*//;
if ( ($db eq "DEPS") and
((! $oldDeps{$name}) or ($oldDeps{$name} ne $depends{$name})) ) {
$hasnew = 1;
} elsif ($db eq "REPO") {
$du{$match} = (-s $match);
$md5sums{$match} = digest_file_hex($match,"MD5");
if (! $descrip{$name}) {$descrip{$name} = "N.A."};
if (! $flags{$name}) {$flags{$name} = "no:no:no"};
if (! $oldFlags{$name}) { $hasnew = 1; }
}
}
return unless ($hasnew == 1);
my ($pname, $dbO, $oname, $pdeps, $desc, $du, $md5, $ppr);
my $count = 0; my ($pver, $url, $date); # needed for the html index
my $dict = ($db eq "DEPS") ? "depends" : "flags";
open (my $fh, ">PKG$db.new");
foreach my $mp (sort keys %$dict) {
$name = $mp; $name =~ s/#.*//;
if ($db eq "REPO") {
printf $fh "%-s:%-s:%-s:%-s:%-s\n", $name, $du{$mp},
$md5sums{$mp}, $descrip{$name}, $flags{$name};
} elsif ($db eq "DEPS") {
printf $fh "%-30s:%s\n", $name, $depends{$name};
foreach my $db ("PKGREPO", "PKGDEPS", "index.html") {
my $firstrun = 0; my $dbNew;
my $status = "+ Generating ";
if ($db eq "PKGREPO") {
$status .= "repository\n";
} elsif ($db eq "PKGDEPS") {
$status .= "dependencies\n";
} else {
$status .= "html index\n";
}
print $status;
open (my $dbOld, "$db") or $firstrun=1;
if ( ($firstrun == 1) and ($db eq "index.html") ) {
printheader();
rename($db, "$db.new");
open ($dbNew, ">>$db.new");
} else {
open ($dbNew, ">$db.new");
}
PACKAGE: foreach my $p (sort @packages) {
$count++ if ($db eq "index.html");
$pname = $p; $pver = $p;
$pname =~ s/#.*//; $pver =~ s/.*#(.*)\.pkg\.tar\.*/$1/;
if ($db eq "PKGREPO") {
$du = (-s $p);
$md5 = digest_file_hex($p,"MD5");
$desc = (! $descrip{$pname}) ? "N.A." : $descrip{$pname};
$ppr = (! $flags{$pname}) ? "no:no:no" : $flags{$pname};
printf $dbNew "%-s:%-s:%-s:%-s:%-s\n",
$p, $du, $md5, $desc, $ppr if ($firstrun == 1);
} elsif ($db eq "PKGDEPS") {
$pdeps = (! $depends{$pname}) ? "" : $depends{$pname};
printf $dbNew "%-30s:%s\n", $pname, $pdeps if ($firstrun == 1);
} else {
$date = isotime( (stat($p))[9], 1);
$url = $p;
$url =~ s/\#/\%23/;
htmlrow($dbNew,$parity{($count % 2)},$pname,$url,$pver,
$descrip{$pname},$date) if ($firstrun == 1);
}
next PACKAGE if ($firstrun == 1);
while ($dbO = <$dbOld>) {
chomp($dbO);
if (($db eq "index.html") and ($dbO !~ m/^<tr class/)) {
# Try to ensure that header lines are copied verbatim,
# by exploiting the alphabetical sorting below.
# Not guaranteed to work with every locale!
$oname = " 0";
} else {
# should be able to extract the old pkg name from this line
$oname = $dbO;
$oname =~ s/\s*\:.*// if ($db eq "PKGDEPS");
$oname =~ s/(.*)\#.*pkg\.tar.*/$1/ if ($db eq "PKGREPO");
$oname =~ s/.*a href="(.*)\%23.*/$1/ if ($db eq "index.html");
$count++ if ($db eq "index.html");
}
if ($oname lt $pname) {
print $dbNew "$dbO\n";
} elsif (($oname ge $pname) and ($db eq "PKGREPO")) {
printf $dbNew "%-s:%-s:%-s:%-s:%-s\n", $p, $du, $md5, $desc, $ppr;
} elsif (($oname ge $pname) and ($db eq "PKGDEPS")) {
printf $dbNew "%-30s:%s\n", $pname, $pdeps
} else {
# either overwrite the old entry in the html index,
# or insert this entry before the first line that
# would come after $p when sorted alphabetically.
htmlrow($dbNew,$parity{($count % 2)},$pname,$url,$pver,
$descrip{$pname},$date);
}
print $dbNew "$dbO\n" if ($oname gt $pname);
last if ($oname ge $pname);
}
}
close ($fh);
rename("PKG$db.new", "PKG$db");
while ( ($firstrun != 1) and ($dbO = <$dbOld>) ) {
if ($db eq "index.html") {
$count++;
$dbO =~ s/class="(even|odd)"/class="$parity{($count % 2)}"/;
}
print $dbNew $dbO;
}
close($dbNew);
($firstrun == 1) or close($dbOld);
rename("$db.new", "$db");
printfooter($count) if (($firstrun == 1) and ($db eq "index.html"));
}
}
######################## full repository ########################
# generate dependency map or the repository/index page
sub pkg_dir {
my $db = shift; my %seen;
my $status = "+ Generating ";
$status .= ($db eq "DEPS") ? "dependencies\n" : "repository\n";
print $status;
open (my $fh, ">PKG$db");
if ($db eq "DEPS") {
foreach my $name (@packages) {
$name =~ s/#.*//; next if ($seen{$name});
if (($depends{$name}) and ($depends{$name} ne "")) {
printf $fh "%-30s : %-s\n", $name, $depends{$name};
}
$seen{$name} = 1;
print "+ Generating dependencies\n";
open (my $iD, ">PKGDEPS");
print "+ Generating repository\n";
open (my $iR, ">PKGREPO");
printheader();
my $count = 0;
open (my $ih, '>>index.html');
foreach my $p (@packages) {
chomp($p);
my $date = isotime( (stat($p))[9], 1);
$count++;
my ($name, $version, $url) = ($p, $p, $p);
$name =~ s/\#.*//;
$version =~ s/^.*\#//;
$version =~ s/\.pkg\.tar\.[gbx]z*//;
$url =~ s/\#/\%23/;
if (($depends{$name}) and ($depends{$name} ne "")) {
printf $iD "%-30s : %-s\n", $name, $depends{$name};
}
} elsif ($db eq "REPO") {
our $parity = "odd";
my $count = 0;
printheader();
open (my $ih, '>>index.html');
foreach my $p (@packages) {
chomp($p);
my $date = (stat($p))[9];
$count++;
my ($name, $version, $url) = ($p, $p, $p);
$name =~ s/#.*//;
$version =~ s/^.*\#//;
$version =~ s/\.pkg\.tar\.[gbx]z*//;
$url =~ s/\#/\%23/;
my $du = (-s $p);
my $md5 = digest_file_hex($p,"MD5");
$md5 =~ s/ .*$|\n//g;
if (! $descrip{$name}) {$descrip{$name} = "N.A.";}
if (! $flags{$name}) { $flags{$name} = "no:no:no"; }
printf $fh "%-s:%-s:%-s:%-s:%-s\n", $p,$du,$md5,$descrip{$name},$flags{$name};
print $ih "<tr class=\"$parity\">";
print $ih "<td>$name</td>";
print $ih "<td><a href=\"$url\">$version</a></td>";
print $ih "<td>$descrip{$name}</td>";
print $ih "<td>" . isotime($date, 1) . "</td>";
print $ih "</tr>\n";
if ($parity eq "odd") { $parity = "even"; }
else { $parity = "odd"; }
}
close $ih;
printfooter($count);
my $du = (-s $p);
my $md5 = digest_file_hex($p,"MD5");
if (! $descrip{$name}) {$descrip{$name} = "N.A.";}
if (! $flags{$name}) { $flags{$name} = "no:no:no"; }
printf $iR "%-s:%-s:%-s:%-s:%-s\n", $p,$du,$md5,$descrip{$name},$flags{$name};
htmlrow($ih,$parity{($count % 2)},$name,$url, $version,$descrip{$name},$date);
}
close $fh;
close($ih);
printfooter($count);
close($iR);
close($iD);
}
# generate README file
sub pkgread {
# consolidate all the README and install scripts for the available packages
sub pkgreadscripts {
print "+ Generating README\n";
open (my $fh, '>PKGREAD');
print $fh "# README files for repository. Do NOT remove this line.\n";
foreach my $name (@packages) {
$name =~ s/#.*//;
if (-f "$path{$name}/README"){
print $fh "##### PKGREADME: $name\n";
open(my $readme, "$path{$name}/README");
while (<$readme>){ print $fh $_; }
close($readme);
}
}
close $fh;
}
open (my $fR, '>PKGREAD');
print $fR "# README files for repository. Do NOT remove this line.\n";
# generate pre-install scripts file
sub pkginst {
print "+ Generating scripts\n";
open (my $fh, '>PKGINST');
print $fh '#!/usr/bin/env bash
open (my $fS, '>PKGINST');
print $fS '#!/usr/bin/env bash
#
# PKGINST: pre- and post-install scripts for CRUX packages
#
run_script() {
case "$1" in
case "$1" in
';
foreach my $name (@packages) {
$name =~ s/#.*//;
foreach my $name (sort @dirlist) {
$name =~ s/\#.*//;
if (-f "$path{$name}/README"){
print $fR "##### PKGREADME: $name\n";
open(my $readme, "$path{$name}/README");
while (<$readme>){ print $fR $_; }
close($readme);
}
foreach my $when ("pre", "post") {
if (-f "$path{$name}/${when}-install"){
print $fh "$name.$when)\n";
open(my $rs, "$path{$name}/${when}-install");
while (<$rs>){
chomp;
if (! m/^#!.*sh/) { print $fh " $_\n"; }
}
close($rs);
print $fh " ;;\n";
}
if (-f "$path{$name}/${when}-install"){
print $fS " $name.$when)\n";
open(my $rs, "$path{$name}/${when}-install");
while (<$rs>){
chomp;
print $fS " $_\n"
unless (m/^\#(!.*sh|\s*EOF|\s*End)/);
}
close($rs);
print $fS " ;;\n";
}
}
}
print $fh "esac\n}\n\n";
print $fh '[ "$1" ] && [[ "$2" == @(pre|post) ]] && run_script "$1.$2"';
print $fh "\n";
close $fh;
print $fS " esac\n}\n\n";
print $fS '[ "$1" ] && [[ "$2" == @(pre|post) ]] && run_script "$1.$2"';
print $fS "\n";
close $fS;
close $fR;
}
######################## html index subs ########################
sub printheader {
@ -294,9 +296,7 @@ EOH
print $ih " <h2>$title</h2>\n";
if ($header) {
open(FILE, $header) or die "Couldn't open header file";
while (<FILE>) {
print $ih " " . $_;
}
while (<FILE>) { print $ih " " . $_; }
close(FILE);
}
@ -307,6 +307,13 @@ EOH
close($ih);
}
sub htmlrow {
my ($ih, $odd, $name, $url, $version, $desc, $date) = @_;
print $ih "<tr class=\"$odd\"><td>$name</td>";
print $ih "<td><a href=\"$url\">$version</a></td>";
print $ih "<td>$desc</td><td>$date</td></tr>\n";
}
sub printfooter {
my $count = $_[0];
open (my $ih, '>>index.html');
@ -314,9 +321,7 @@ sub printfooter {
print $ih " <p><b>$count packages</b></p>\n";
if ($footer) {
open(FILE, $footer) or die "Couldn't open footer file";
while (<FILE>) {
print $ih " " . $_;
}
while (<FILE>) { print $ih " " . $_; }
close(FILE);
}
print $ih " <p><i>Generated by <a href=\"http://www.varlock.com\">pkg-repgen</a> on " . isotime() . ".</i></p>\n";