tools/httpup
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

import httpup 0.4.0f

Browse Source
This commit is contained in:
Johannes Winkelmann 2005-11-09 21:44:34 +00:00
commit 18c22c5a41
25 changed files with 2997 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
AUTHORS Normal file
View File

@ -0,0 +1,9 @@
AUTHORS:
Johannes Winkelmann <jw@tks6.net>
md5 code is written by Christophe Devine
THANKS:
Simone Rota for testing and bug reports
Jürgen Daubert for testing
Han Boetes for optimizing the repgen script

341
COPYING Normal file
View File

@ -0,0 +1,341 @@
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
Copyright (C) 1989, 1991 Free Software Foundation, Inc.
59 Temple Place - Suite 330
Boston, MA 02111-1307, USA.
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The licenses for most software are designed to take away your
freedom to share and change it. By contrast, the GNU General Public
License is intended to guarantee your freedom to share and change free
software--to make sure the software is free for all its users. This
General Public License applies to most of the Free Software
Foundation's software and to any other program whose authors commit to
using it. (Some other Free Software Foundation software is covered by
the GNU Library General Public License instead.) You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
this service if you wish), that you receive source code or can get it
if you want it, that you can change the software or use pieces of it
in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid
anyone to deny you these rights or to ask you to surrender the rights.
These restrictions translate to certain responsibilities for you if you
distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must give the recipients all the rights that
you have. You must make sure that they, too, receive or can get the
source code. And you must show them these terms so they know their
rights.
We protect your rights with two steps: (1) copyright the software, and
(2) offer you this license which gives you legal permission to copy,
distribute and/or modify the software.
Also, for each author's protection and ours, we want to make certain
that everyone understands that there is no warranty for this free
software. If the software is modified by someone else and passed on, we
want its recipients to know that what they have is not the original, so
that any problems introduced by others will not reflect on the original
authors' reputations.
Finally, any free program is threatened constantly by software
patents. We wish to avoid the danger that redistributors of a free
program will individually obtain patent licenses, in effect making the
program proprietary. To prevent this, we have made it clear that any
patent must be licensed for everyone's free use or not licensed at all.
The precise terms and conditions for copying, distribution and
modification follow.
GNU GENERAL PUBLIC LICENSE
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
0. This License applies to any program or other work which contains
a notice placed by the copyright holder saying it may be distributed
under the terms of this General Public License. The "Program", below,
refers to any such program or work, and a "work based on the Program"
means either the Program or any derivative work under copyright law:
that is to say, a work containing the Program or a portion of it,
either verbatim or with modifications and/or translated into another
language. (Hereinafter, translation is included without limitation in
the term "modification".) Each licensee is addressed as "you".
Activities other than copying, distribution and modification are not
covered by this License; they are outside its scope. The act of
running the Program is not restricted, and the output from the Program
is covered only if its contents constitute a work based on the
Program (independent of having been made by running the Program).
Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program's
source code as you receive it, in any medium, provided that you
conspicuously and appropriately publish on each copy an appropriate
copyright notice and disclaimer of warranty; keep intact all the
notices that refer to this License and to the absence of any warranty;
and give any other recipients of the Program a copy of this License
along with the Program.
You may charge a fee for the physical act of transferring a copy, and
you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion
of it, thus forming a work based on the Program, and copy and
distribute such modifications or work under the terms of Section 1
above, provided that you also meet all of these conditions:
a) You must cause the modified files to carry prominent notices
stating that you changed the files and the date of any change.
b) You must cause any work that you distribute or publish, that in
whole or in part contains or is derived from the Program or any
part thereof, to be licensed as a whole at no charge to all third
parties under the terms of this License.
c) If the modified program normally reads commands interactively
when run, you must cause it, when started running for such
interactive use in the most ordinary way, to print or display an
announcement including an appropriate copyright notice and a
notice that there is no warranty (or else, saying that you provide
a warranty) and that users may redistribute the program under
these conditions, and telling the user how to view a copy of this
License. (Exception: if the Program itself is interactive but
does not normally print such an announcement, your work based on
the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If
identifiable sections of that work are not derived from the Program,
and can be reasonably considered independent and separate works in
themselves, then this License, and its terms, do not apply to those
sections when you distribute them as separate works. But when you
distribute the same sections as part of a whole which is a work based
on the Program, the distribution of the whole must be on the terms of
this License, whose permissions for other licensees extend to the
entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest
your rights to work written entirely by you; rather, the intent is to
exercise the right to control the distribution of derivative or
collective works based on the Program.
In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.
3. You may copy and distribute the Program (or a work based on it,
under Section 2) in object code or executable form under the terms of
Sections 1 and 2 above provided that you also do one of the following:
a) Accompany it with the complete corresponding machine-readable
source code, which must be distributed under the terms of Sections
1 and 2 above on a medium customarily used for software interchange; or,
b) Accompany it with a written offer, valid for at least three
years, to give any third party, for a charge no more than your
cost of physically performing source distribution, a complete
machine-readable copy of the corresponding source code, to be
distributed under the terms of Sections 1 and 2 above on a medium
customarily used for software interchange; or,
c) Accompany it with the information you received as to the offer
to distribute corresponding source code. (This alternative is
allowed only for noncommercial distribution and only if you
received the program in object code or executable form with such
an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for
making modifications to it. For an executable work, complete source
code means all the source code for all modules it contains, plus any
associated interface definition files, plus the scripts used to
control compilation and installation of the executable. However, as a
special exception, the source code distributed need not include
anything that is normally distributed (in either source or binary
form) with the major components (compiler, kernel, and so on) of the
operating system on which the executable runs, unless that component
itself accompanies the executable.
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent
access to copy the source code from the same place counts as
distribution of the source code, even though third parties are not
compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program
except as expressly provided under this License. Any attempt
otherwise to copy, modify, sublicense or distribute the Program is
void, and will automatically terminate your rights under this License.
However, parties who have received copies, or rights, from you under
this License will not have their licenses terminated so long as such
parties remain in full compliance.
5. You are not required to accept this License, since you have not
signed it. However, nothing else grants you permission to modify or
distribute the Program or its derivative works. These actions are
prohibited by law if you do not accept this License. Therefore, by
modifying or distributing the Program (or any work based on the
Program), you indicate your acceptance of this License to do so, and
all its terms and conditions for copying, distributing or modifying
the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the
Program), the recipient automatically receives a license from the
original licensor to copy, distribute or modify the Program subject to
these terms and conditions. You may not impose any further
restrictions on the recipients' exercise of the rights granted herein.
You are not responsible for enforcing compliance by third parties to
this License.
7. If, as a consequence of a court judgment or allegation of patent
infringement or for any other reason (not limited to patent issues),
conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot
distribute so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you
may not distribute the Program at all. For example, if a patent
license would not permit royalty-free redistribution of the Program by
all those who receive copies directly or indirectly through you, then
the only way you could satisfy both it and this License would be to
refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under
any particular circumstance, the balance of the section is intended to
apply and the section as a whole is intended to apply in other
circumstances.
It is not the purpose of this section to induce you to infringe any
patents or other property right claims or to contest validity of any
such claims; this section has the sole purpose of protecting the
integrity of the free software distribution system, which is
implemented by public license practices. Many people have made
generous contributions to the wide range of software distributed
through that system in reliance on consistent application of that
system; it is up to the author/donor to decide if he or she is willing
to distribute software through any other system and a licensee cannot
impose that choice.
This section is intended to make thoroughly clear what is believed to
be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in
certain countries either by patents or by copyrighted interfaces, the
original copyright holder who places the Program under this License
may add an explicit geographical distribution limitation excluding
those countries, so that distribution is permitted only in or among
countries not thus excluded. In such case, this License incorporates
the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions
of the General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the Program
specifies a version number of this License which applies to it and "any
later version", you have the option of following the terms and conditions
either of that version or of any later version published by the Free
Software Foundation. If the Program does not specify a version number of
this License, you may choose any version ever published by the Free Software
Foundation.
10. If you wish to incorporate parts of the Program into other free
programs whose distribution conditions are different, write to the author
to ask for permission. For software which is copyrighted by the Free
Software Foundation, write to the Free Software Foundation; we sometimes
make exceptions for this. Our decision will be guided by the two goals
of preserving the free status of all derivatives of our free software and
of promoting the sharing and reuse of software generally.
NO WARRANTY
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN
OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS
TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE
PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
convey the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) 19yy <name of author>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; see the file COPYING. If not, write to
the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
Boston, MA 02111-1307, USA.
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this
when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) 19yy name of author
Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, the commands you use may
be called something other than `show w' and `show c'; they could even be
mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your
school, if any, to sign a "copyright disclaimer" for the program, if
necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program
`Gnomovision' (which makes passes at compilers) written by James Hacker.
<signature of Ty Coon>, 1 April 1989
Ty Coon, President of Vice
This General Public License does not permit incorporating your program into
proprietary programs. If your program is a subroutine library, you may
consider it more useful to permit linking proprietary applications with the
library. If this is what you want to do, use the GNU Library General
Public License instead of this License.

76
ChangeLog Normal file
View File

@ -0,0 +1,76 @@
* 0.4.0f 22.09.2005 Johannes Winkelmann
- remove deflate option again
* 0.4.0e 20.09.2005 Johannes Winkelmann
- Add deflate option
- Set timeout to 30s
* 0.4.0d 03.05.2005 Johannes Winkelmann
- fallback when argument list too long
* 0.4.0b 03.05.2005 Johannes Winkelmann
- fix two bugs in -repgen with ignore expressions
* 0.4.0a 02.04.2005 Johannes Winkelmann
- Fix a nasty bug when write permissions are missing
* 0.4.0 04.02.2005 Johannes Winkelmann
- Initial import of argparser
- remove 'mirror' command
- 'copy' command requires 2 arguments
- support nondefault REPO file name
- include new httpup-repgen: new ignore syntax
- refactorings
- add --verify-md5 to verify md5sum
- nonzero return value if md5sum verification failed
- merge Han's many changes to httpup-repgen
* 0.3.2 28.07.2004 Johannes Winkelmann
- New commands: copy (no additional files), mirror (keep REPO file)
- Disable debug message in main.cpp (thanks Simone)
* 0.3.1 15.04.2004 Johannes Winkelmann
- fix segfault if no .httpup-url found
- remove debug output
- add missing include
- fix substring checkout bug
* 0.3.0 14.04.2004 Johannes Winkelmann
- Enable http 302 redirection
* 0.2.94 07.04.2004 Johannes Winkelmann
- add detection for malicious file names in remote repo files (".." at least)
- update man page
* 0.2.93 05.04.2004 Johannes Winkelmann
- fix config parser
- fix bug in proxy auth (curl <-> std::string)
* 0.2.92 03.04.2004 Johannes Winkelmann
- rename REPO.CURRENT to .httpup-repo.current (stay backward compatible for
0.3.0)
- Remove files with directories, warn if a files should replace a directory
- support for proxy server and proxy authentication
- make 'list' recursive
* 0.2.91 28.03.2004 Johannes Winkelmann
- use ~/.netrc for supplying username/password to proxy
* 0.2.90 26.03.2004 Johannes Winkelmann
- Major refactoring; make it partly object oriented :-)
- Use pwd if no target directory specified
- allow sync without arguments -> use urlinfo in pwd
- allow subtree sync
* 0.2.2 04.10.2003 Johannes Winkelmann
- Fix ugly bug caused by temporary c_str() references
* 0.2.1 lost
* 0.2 27.06.2003 Johannes Winkelmann
- Bugfix when deleting files
* 0.1 23.06.2003 Johannes Winkelmann
- initial release

43
Makefile Normal file
View File

@ -0,0 +1,43 @@
all: httpup
############################################################################
###
## Configuration
#
NAME=httpup
VERSION="0.4.0f"
CXX=g++
CXXFLAGS=-Wall -ansi -pedantic -DMF_VERSION='${VERSION}'
LDFLAGS=-lcurl
objects=httpupargparser.o argparser.o main.o httpup.o \
fileutils.o md5.o configparser.o
httpupargparser.o: httpupargparser.cpp httpupargparser.h
argparser.o: argparser.cpp argparser.h
main.o: main.cpp
httpup.o: httpup.cpp httpup.h
fileutils.o: fileutils.cpp fileutils.h
md5.o: md5.cpp md5.h
configparser.o: configparser.cpp configparser.h
############################################################################
$(objects): %.o: %.cpp
$(CXX) -c $(CXXFLAGS) $< -o $@
httpup: $(objects) *.cpp *.h
g++ -o httpup $(objects) $(LDFLAGS)
clean:
rm -f httpup $(objects)
dist:
rm -rf ${NAME}-${VERSION}
mkdir ${NAME}-${VERSION}
cp *.cpp *.h Makefile AUTHORS COPYING ChangeLog README TODO *.8 \
httpup-repgen* httpup.conf* ${NAME}-${VERSION}
tar cvzf ${NAME}-${VERSION}.tar.gz ${NAME}-${VERSION}
rm -rf ${NAME}-${VERSION}

6
README Normal file
View File

@ -0,0 +1,6 @@
This is httpup, and one way synchronize tool over http. It depends on libcurl
and was written on linux but it should build on any platform supported by
libcurl.
If you find bugs or run into any problems feel free to contact me by e-mail
Johannes, jw@tks6.net

35
TODO Normal file
View File

@ -0,0 +1,35 @@
* add support for tarball download
- 1. download tarball if no directory exists or --tarball
2. decompress and create .httpup-current
3. download REPO
4. sync as usual
- make sure we create a .httpup-current to ensure all files downloaded are
md5sum checked or removed
* Think about permission preservation
* man page
- httpup.conf
* Feature:
- add a -k (keep) feature
- better return for findDiff()
- don0t create dir if no matches
* Concept:
change it to use the following syntax:
httpup diff [target dir] - show changed/added/removed files
if baseurl is omitted, it is read from .httpup-urlinfo
if target dir is omitted, httpup looks for an .httpup-urlinfo file in the
current working directory. In order to get such a file, use httpup sync -k
(keep).
* Refactoring:
- make more modular (network handler, file handler)
- refactor main
* Bugs
- handle replacement of files with directories (and the other way around)

420
argparser.cpp Normal file
View File

@ -0,0 +1,420 @@
////////////////////////////////////////////////////////////////////////
// FILE: argparser.cpp
// AUTHOR: Johannes Winkelmann, jw@tks6.net
// COPYRIGHT: (c) 2004 by Johannes Winkelmann
// ---------------------------------------------------------------------
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
////////////////////////////////////////////////////////////////////////
#include <iostream>
#include <sstream>
#include <cassert>
#include <libgen.h>
#include "argparser.h"
using namespace std;
ArgParser::ArgParser()
: m_cmdIdCounter(0),
m_optIdCounter(0)
{
}
ArgParser::~ArgParser()
{
map<int, Option*>::iterator oit = m_options.begin();
for (; oit != m_options.end(); ++oit) {
delete oit->second;
}
map<string, Command*>::iterator cit = m_commands.begin();
for (; cit != m_commands.end(); ++cit) {
delete cit->second;
}
}
int ArgParser::addCommand(APCmd& cmd,
const std::string& name,
const std::string& description,
ArgNumberCheck argNumberCheck,
int argNumber,
const std::string& otherArguments)
{
Command* command = new Command;
++m_cmdIdCounter;
cmd.id = m_cmdIdCounter;
command->apCmd = &cmd;
command->id = m_cmdIdCounter;
command->name = name;
command->argNumber = argNumber;
command->argNumberCheck = argNumberCheck;
command->description = description;
command->otherArguments = otherArguments;
m_commands[name] = command;
m_commandIdMap[cmd.id] = command;
APCmd apcmd;
apcmd.id = m_cmdIdCounter;
PREDEFINED_CMD_HELP.init("help", 'h', "Print this help message");
// add predefined commands
addOption(cmd, PREDEFINED_CMD_HELP, false);
return 0;
}
int ArgParser::addOption(const APCmd& commandKey,
APOpt& key,
bool required)
{
// TODO: check for null cmd
if (m_commandIdMap.find(commandKey.id) == m_commandIdMap.end()) {
return -1;
}
Option* o = 0;
if (key.id != -1 && m_options.find(key.id) != m_options.end()) {
o = m_options.find(key.id)->second;
}
if (!o) {
assert(key.m_initialized == true);
o = new Option();
++m_optIdCounter;
key.id = m_optIdCounter;
o->id = key.id;
o->description = key.m_description;
o->requiresValue = key.m_valueRequired;
o->shortName = key.m_shortName;
o->longName = key.m_longName;
o->valueName = key.m_valueName;
if (key.m_shortName != 0) {
m_optionsByShortName[key.m_shortName] = o;
}
if (key.m_longName != "") {
m_optionsByLongName[key.m_longName] = o;
}
m_options[key.id] = o;
}
Command* cmd = m_commandIdMap[commandKey.id];
if (required) {
cmd->mandatoryOptions[key.id] = o;
} else {
cmd->options[key.id] = o;
}
return 0;
}
void ArgParser::parse(int argc, char** argv)
{
bool commandFound = false;
string command = "";
Command* cmd = 0;
int cmdPos = 0;
m_appName = basename(argv[0]);
for (int i = 1; i < argc; ++i) {
if (argv[i][0] != '-') {
if (!commandFound) {
if (m_commands.find(argv[i]) == m_commands.end()) {
parseError("Non option / Non command argument '" +
string(argv[i]) + "'");
}
cmd = m_commands[argv[i]];
m_command.id = cmd->apCmd->id;
commandFound = true;
cmdPos = i;
break;
}
} else {
// TODO: add proper handling for global options
string arg = argv[i];
if (arg == "-h" || arg == "--help") {
cout << generateUsage() << endl;
exit(0);
}
}
}
if (!commandFound) {
parseError("No command used");
exit(-1);
}
for (int i = 1; i < argc; ++i) {
if (i == cmdPos) {
continue;
}
if (argv[i][0] == '-') {
if (argv[i][1] == '\0') {
parseError("Illegal token: '-'", cmd->name);
} else if (argv[i][1] == '-') {
char* valPtr = strchr(argv[i]+2, '=');
if (valPtr) {
*valPtr = '\0';
++valPtr;
}
if (m_optionsByLongName.find(argv[i]+2) ==
m_optionsByLongName.end()) {
parseError("unknown option:" + string(argv[i]+2),
cmd->name);
}
Option* o = m_optionsByLongName[argv[i]+2];
string val = "";
if (o->requiresValue) {
if (valPtr == NULL || *valPtr == 0) {
parseError("Value required for option '" +
string(argv[i]+2), cmd->name);
} else {
val = valPtr;
}
}
m_setOptions[o->id] = val;
} else {
if (argv[i][2] != '\0') {
parseError("invalid short option '" +
string(argv[i]+1) + "'", cmd->name);
}
if (m_optionsByShortName.find(argv[i][1]) ==
m_optionsByShortName.end()) {
parseError("unknown short option:" + string(argv[i]+1),
cmd->name);
}
Option* o = m_optionsByShortName[argv[i][1]];
string val = "";
if (o->requiresValue) {
if (i+1 == argc) {
parseError("Option required for option '" +
string(argv[i]+1), cmd->name);
} else {
val = argv[i+1];
++i;
}
}
m_setOptions[o->id] = val;
}
} else {
m_otherArguments.push_back(string(argv[i]));
}
}
if (isSet(PREDEFINED_CMD_HELP)) {
cout << generateHelpForCommand(cmd->name) << endl;
exit(0);
} else {
// make sure all required options of a command are set
std::map<int, Option*>::iterator it;
it = cmd->mandatoryOptions.begin();
for (; it != cmd->mandatoryOptions.end(); ++it) {
if (!isSet(it->second->id)) {
parseError("Command '" + cmd->name +
"' requires option " +
string("-") + it->second->shortName +
string(" | ") +
string("--") + it->second->longName + " not found",
cmd->name);
}
}
}
switch (cmd->argNumberCheck)
{
case EQ:
if (m_otherArguments.size() != cmd->argNumber) {
ostringstream ostr;
ostr << cmd->name
<< " takes exactly "
<< cmd->argNumber
<< (cmd->argNumber == 1 ? " argument." : " arguments.");
parseError(ostr.str(), cmd->name);
}
break;
case MIN:
if (m_otherArguments.size() < cmd->argNumber) {
ostringstream ostr;
ostr << cmd->name
<< " takes at least "
<< cmd->argNumber
<< (cmd->argNumber == 1 ? " argument." : " arguments.");
parseError(ostr.str(), cmd->name);
}
break;
case MAX:
if (m_otherArguments.size() > cmd->argNumber) {
ostringstream ostr;
ostr << cmd->name
<< " takes at most "
<< cmd->argNumber
<< (cmd->argNumber == 1 ? " argument." : " arguments.");
parseError(ostr.str(), cmd->name);
}
break;
case NONE:
default:
break;
}
}
void ArgParser::parseError(const string& error, const string& cmd) const
{
cerr << "Parse error: " << error << endl;
if (cmd != "") {
cerr << generateHelpForCommand(cmd) << endl;
} else {
cerr << generateUsage() << endl;
}
exit(-1);
}
ArgParser::APCmd ArgParser::command() const
{
return m_command;
}
bool ArgParser::isSet(const APOpt& key) const
{
return isSet(key.id);
}
bool ArgParser::isSet(int key) const
{
return m_setOptions.find(key) != m_setOptions.end();
}
std::string ArgParser::getOptionValue(const APOpt& key) const
{
return m_setOptions.find(key.id)->second;
}
std::string ArgParser::appName() const
{
return m_appName;
}
std::string ArgParser::generateHelpForCommand(const std::string& command) const
{
std::map<std::string, Command*>::const_iterator cit =
m_commands.find(command);
if (cit == m_commands.end()) {
return "";
}
const Command * const cmd = cit->second;
string help = "";;
help += "command '" + cmd->name + " " + cmd->otherArguments + "'\n";
help += " " + cmd->description;
help += "\n\n";
std::map<int, Option*>::const_iterator it =
it = cmd->mandatoryOptions.begin();
if (it != cmd->mandatoryOptions.end()) {
help += " Required: \n";
for (; it != cmd->mandatoryOptions.end(); ++it) {
help += generateOptionString(it->second);
}
}
it = cmd->options.begin();
if (it != cmd->options.end()) {
help += " Optional: \n";
for (; it != cmd->options.end(); ++it) {
help += generateOptionString(it->second);
}
}
return help;
}
string ArgParser::generateOptionString(Option* o) const
{
string help = " ";
if (o->shortName) {
help += "-";
help += o->shortName;
if (o->requiresValue && o->valueName != "") {
help += " " + o->valueName;
}
help += " | ";
}
if (o->longName != "") {
help += "--";
help += o->longName;
if (o->requiresValue && o->valueName != "") {
help += "=" + o->valueName;
}
help += " ";
help += o->description;
help += "\n";
}
return help;
}
std::string ArgParser::generateUsage() const
{
string usage = getAppIdentification() +
"USAGE: " + m_appName +
" [OPTIONS] command <arguments>\n\n";
usage += " Where command is one of the following:\n";
std::map<std::string, Command*>::const_iterator it;
it = m_commands.begin();
for (; it != m_commands.end(); ++it) {
usage += " " + it->first + "\t\t" +
it->second->description + "\n";
}
return usage;
}
const std::vector<std::string>& ArgParser::otherArguments() const
{
return m_otherArguments;
}

242
argparser.h Normal file
View File

@ -0,0 +1,242 @@
////////////////////////////////////////////////////////////////////////
// FILE: argparser.h
// AUTHOR: Johannes Winkelmann, jw@tks6.net
// COPYRIGHT: (c) 2004 by Johannes Winkelmann
// ---------------------------------------------------------------------
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
////////////////////////////////////////////////////////////////////////
#ifndef _ARGPARSER_H_
#define _ARGPARSER_H_
#include <map>
#include <vector>
#include <string>
// TODO:
// -- important
// - allow multiple occurences of arguments:
// prt-get --config-append="..." --config-append="..."
// - Allow global --help, --usage
// - check for duplicate entries
// -- nice to have
// 2. allow optional values for args, like --with-gtk[=DIR]
// 4. improve predefined commands: allow descriptions even for such
// 7. Add predefined --version, show in usage; add Contact text
// 8. Allow disabling of predefined options
// 9. make parseError more convenient (passing cmd->name all over...)
class APOpt;
class APCmd;
/**
* \brief argument parser class
*
* Yet another argument parser, meant to speed up development of new
* applications. Its focus is on being object oriented and safe to use
*/
class ArgParser
{
public:
/**
* types of argument number checking
*/
enum ArgNumberCheck { NONE, MIN, EQ, MAX };
/*
* APOpt and APCmd are the classes used in client code, to make
* efficient comparison of the selected command.
*
* In addition, it's currently mainly for programs which use a
* command syntax, much like CVS
*/
class APOpt
{
public:
friend class ArgParser;
APOpt() : id(-1), m_initialized(false) {}
bool operator ==(const APOpt& other) const { return other.id == id; }
void init(const std::string& longName,
char shortName,
const std::string& description,
const bool valueRequired=false,
const std::string& valueName="") {
m_initialized = true;
m_longName = longName;
m_shortName = shortName;
m_description = description;
m_valueRequired = valueRequired;
m_valueName = valueName;
}
private:
int id;
std::string m_longName;
char m_shortName;
std::string m_description;
bool m_valueRequired;
std::string m_valueName;
bool m_initialized;
};
class APCmd
{
public:
friend class ArgParser;
APCmd() : id(-1) {}
bool operator ==(const APCmd& other) const { return other.id == id; }
private:
int id;
};
private:
// internal representation of options and commands
class Option
{
public:
int id;
std::string description;
char shortName;
std::string longName;
bool requiresValue;
std::string valueName;
};
class Command
{
public:
int id;
std::string name;
std::string description;
ArgNumberCheck argNumberCheck;
unsigned int argNumber;
std::string otherArguments;
std::map<int, Option*> mandatoryOptions;
std::map<int, Option*> options;
ArgParser::APCmd* apCmd;
};
public:
ArgParser();
virtual ~ArgParser();
/**
* add a command
*
* \param cmd a reference to the command; use it to compare the actually selected command against this one after parsing
* \param name the name of the command to be parsed from the command line
* \param description a description, used for the help screens
* \param argNumberCheck what kind of argument number checking
* \param argNumber optional number of arguments
* \param otherOptions value to display in the help screen for following (non option) arguments
*/
int addCommand(APCmd& cmd,
const std::string& name,
const std::string& description,
ArgNumberCheck argNumberCheck,
const int argNumber=-1,
const std::string& otherArguments="");
/**
* add an option to a command - this will fail with an assertion
* of \a key has not been initialized using init()
*
* \param cmd the command to add an option to
* \param key the option reference; use it to check for certain options after parsing
* \param required whether this option is required
* \param longName the long name of this command (to be used with '--'); leave it empty if you don't want to use a long option name
* \param shortName the short name of this command (to be used with '-'); pass 0 if you don't want to use a short option name
* \param description the description of this option, to be used in the help screen
* \param valueRequired whether this option requires a value
* \param valueName the name of the value, to be used in the help screen
*/
int addOption(const APCmd& cmd,
APOpt& key,
bool required);
/**
* the actual parsing. Highly recommended :-)
*/
void parse(int argc, char** argv);
/**
* the command which was parsed, to be used to compare against
* actual APCmd obtained from addCommand calls
*/
APCmd command() const;
/**
* the name of the application, from argv[0]
*/
std::string appName() const;
/**
* \return true if \a key is set, false otherwise
*/
bool isSet(const APOpt& key) const;
/**
* \return the value attached to the option \key if any
*/
std::string getOptionValue(const APOpt& key) const;
/**
* the remaining arguments
*/
const std::vector<std::string>& otherArguments() const;
/**
* \return an application identification to be used in the usage
*/
virtual std::string getAppIdentification() const { return ""; }
private:
std::string generateHelpForCommand(const std::string& command) const;
std::string generateUsage() const;
bool isSet(int key) const;
std::string generateOptionString(Option* o) const;
void parseError(const std::string& error, const std::string& cmd="") const;
std::map<std::string, Command*> m_commands;
std::map<int, Command*> m_commandIdMap;
std::map<int, Option*> m_options;
std::map<char, Option*> m_optionsByShortName;
std::map<std::string, Option*> m_optionsByLongName;
std::map<int, std::string> m_setOptions;
std::vector<std::string> m_otherArguments;
APCmd m_command;
std::string m_appName;
int m_cmdIdCounter;
int m_optIdCounter;
APOpt PREDEFINED_CMD_HELP;
};
#endif /* _ARGPARSER_H_ */

70
configparser.cpp Normal file
View File

@ -0,0 +1,70 @@
////////////////////////////////////////////////////////////////////////
// FILE: configparser.cpp
// AUTHOR: Johannes Winkelmann, jw@tks6.net
// COPYRIGHT: (c) 2002-2005 by Johannes Winkelmann
// ---------------------------------------------------------------------
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
////////////////////////////////////////////////////////////////////////
#include <iostream>
#include "configparser.h"
using namespace std;
int ConfigParser::parseConfig(const std::string& fileName,
Config& config)
{
FILE* fp = fopen(fileName.c_str(), "r");
if (!fp) {
return -1;
}
char line[512];
string s;
while (fgets(line, 512, fp)) {
if (line[strlen(line)-1] == '\n') {
line[strlen(line)-1] = '\0';
}
s = line;
string::size_type pos = s.find("#");
if (pos != string::npos) {
s = s.substr(0, pos);
}
if (s.length() > 10) {
string key = s.substr(0, 10);
string val = stripWhiteSpace(s.substr(10));
if (key == "proxy_host") {
config.proxyHost = val;
} else if (s.substr(0, 10) == "proxy_port") {
config.proxyPort = val;
} else if (s.substr(0, 10) == "proxy_user") {
config.proxyUser = val;
} else if (s.substr(0, 10) == "proxy_pass") {
config.proxyPassword = val;
}
}
}
fclose(fp);
return 0;
}
string ConfigParser::stripWhiteSpace(const string& input)
{
string output = input;
while (isspace(output[0])) {
output = output.substr(1);
}
while (isspace(output[output.length()-1])) {
output = output.substr(0, output.length()-1);
}
return output;
}

35
configparser.h Normal file
View File

@ -0,0 +1,35 @@
////////////////////////////////////////////////////////////////////////
// FILE: configparser.h
// AUTHOR: Johannes Winkelmann, jw@tks6.net
// COPYRIGHT: (c) 2002-2005 by Johannes Winkelmann
// ---------------------------------------------------------------------
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
////////////////////////////////////////////////////////////////////////
#ifndef _CONFIGPARSER_H_
#define _CONFIGPARSER_H_
#include <string>
struct Config
{
Config() : proxyHost(""), proxyPort(""), proxyUser(""), proxyPassword("")
{}
std::string proxyHost;
std::string proxyPort;
std::string proxyUser;
std::string proxyPassword;
};
class ConfigParser
{
public:
static std::string stripWhiteSpace(const std::string& input);
static int parseConfig(const std::string& fileName,
Config& config);
};
#endif /* _CONFIGPARSER_H_ */

173
fileutils.cpp Normal file
View File

@ -0,0 +1,173 @@
////////////////////////////////////////////////////////////////////////
// FILE: fileutils.cpp
// AUTHOR: Johannes Winkelmann, jw@tks6.net
// COPYRIGHT: (c) 2002-2005 by Johannes Winkelmann
// ---------------------------------------------------------------------
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
////////////////////////////////////////////////////////////////////////
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <dirent.h>
#include <iostream>
#include "md5.h"
#include "httpup.h"
#include "fileutils.h"
using namespace std;
int FileUtils::deltree(const char* directory)
{
int ret = 0;
struct stat info;
if (stat(directory, &info)) {
// already removed
return 0;
}
if (!S_ISDIR(info.st_mode)) {
return unlink(directory);
}
DIR* dir = opendir(directory);
struct dirent* entry;
while ((entry = readdir(dir)) != 0) {
if (entry->d_name[0] == '.' &&
(entry->d_name[1] == '.' || entry->d_name[1] == '\0')) {
continue;
}
struct stat info;
stat(entry->d_name, &info);
if (S_ISDIR(info.st_mode)) {
if (deltree(entry->d_name)) {
ret = -1;
}
rmdir(entry->d_name);
} else {
string file = string(directory) + "/" + string(entry->d_name);
if (unlink(file.c_str())) {
ret = -1;
}
}
}
closedir(dir);
if (rmdir(directory)) {
ret = -1;
}
return ret;
}
int FileUtils::mktree(const string& directory)
{
int ret = 0;
size_t pos = 0;
string fName;
while ((pos = directory.find( '/', pos+1)) != string::npos ) {
fName = directory.substr(0, pos);
struct stat info;
if (stat(fName.c_str(), &info)) {
if (mkdir(fName.c_str(), 0755)) {
ret = -1;
}
}
}
return ret;
}
bool FileUtils::fmd5sum(const string& fileName, unsigned char* result)
{
struct md5_context ctx;
unsigned char buffer[1000];
FILE* f = fopen(fileName.c_str(), "r");
if (!f) {
return false;
}
md5_starts( &ctx );
int i = 0;
while( ( i = fread( buffer, 1, sizeof( buffer ), f ) ) > 0 ) {
md5_update( &ctx, buffer, i );
}
fclose(f);
md5_finish( &ctx, result );
return true;
}
void FileUtils::listFiles(const string& target)
{
list<string> files;
string newTarget = target;
if (newTarget != ".") {
if (newTarget[newTarget.length()-1] != '/') {
newTarget += "/";
}
}
string repoFile = newTarget + "/" + HttpUp::REPOCURRENTFILE;
FILE* fp = fopen(repoFile.c_str(), "r");
if (fp) {
char line[512];
while (fgets(line, 512, fp)) {
line[strlen(line)-1] = '\0';
files.push_back(line);
}
fclose(fp);
listFilesRec(newTarget, "", files);
} else {
cerr << "Failed to open " << repoFile << endl;
}
}
void FileUtils::listFilesRec(const string& base,
const string& offset,
list<string>& files)
{
string newOff = offset;
if (newOff.length() > 0) {
newOff += "/";
}
DIR* dir = opendir((base + newOff).c_str());
if (dir) {
struct dirent* d;
string name;
while ((d = readdir(dir))) {
name = d->d_name;
if (name == HttpUp::REPOCURRENTFILE ||
name == HttpUp::URLINFO ||
name == "." || name == "..") {
continue;
}
if (find(files.begin(), files.end(),
newOff + d->d_name) == files.end()) {
cout << "? ";
} else {
cout << "= ";
}
cout << newOff
<< d->d_name << endl;
struct stat buf;
if (stat(((base + newOff) + d->d_name).c_str(), &buf) == 0 &&
S_ISDIR(buf.st_mode)) {
listFilesRec(base, newOff + d->d_name, files);
}
}
closedir(dir);
}
}

32
fileutils.h Normal file
View File

@ -0,0 +1,32 @@
////////////////////////////////////////////////////////////////////////
// FILE: fileutils.h
// AUTHOR: Johannes Winkelmann, jw@tks6.net
// COPYRIGHT: (c) 2002-2005 by Johannes Winkelmann
// ---------------------------------------------------------------------
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
////////////////////////////////////////////////////////////////////////
#ifndef _FILEUTILS_H_
#define _FILEUTILS_H_
#include <string>
#include <list>
class FileUtils
{
public:
static bool fmd5sum(const std::string& fileName, unsigned char* result);
static int deltree(const char* directory);
static int mktree(const std::string& directory);
static void listFiles(const std::string& target);
static void listFilesRec(const std::string& base,
const std::string& offset,
std::list<std::string>& files);
};
#endif /* _FILEUTILS_H_ */

84
httpup-repgen Executable file
View File

@ -0,0 +1,84 @@
#!/bin/sh
# httpup-repgen - One way sync from an http server to a local directory
#
# Copyright 2003-2005 (c) Johannes Winkelmann, # jw@tks6.net
#
# - Filtering code adapted from Per Liden's pkgmk
# - optimized and made portable (sh-compliant) by Han Boetes
# The repo file is place on the server. httpup downloads it,
# makes the update and afterwards moves it to the REPOCURRENTFILE
# which keeps track of the files which have been checked out. The
# REPOCURRENTFILE contains only file names
REPOFILE=REPO
REPOCURRENTFILE=REPO.CURRENT
VERSION=0.8
info()
{
echo $*
}
debug()
{
return # echo $*
}
printUsage()
{
cat << EOF
httpup-repgen $VERSION
Copyright (c) 2003 Johannes Winkelmann
Usage:
httpup-repgen [directory]
EOF
exit -1
}
generateRepoFile()
{
dir=${1:-.}
if [ ! -d $dir ]; then
echo "Can't generate repository for '$dir': No such directory"
exit -2
fi
echo "Generating repository for directory '$dir'"
OLDPWD=$PWD
cd $dir
rm -f $REPOFILE || exit -3
IGNORE_FILE=.httpup-repgen-ignore
if [ -r $HOME/$IGNORE_FILE ]; then
FILTER="grep -E -v -f $HOME/$IGNORE_FILE"
else
FILTER="cat"
fi
if [ -r $IGNORE_FILE ]; then
FILTER_LOCAL="grep -E -v -f $IGNORE_FILE"
else
FILTER_LOCAL="cat"
fi
FILTER_OWN="egrep -v ($REPOFILE|$REPOCURRENTFILE|$IGNORE_FILE)"
find . -type d ! -name . -printf "%P\n"|$FILTER|$FILTER_LOCAL|$FILTER_OWN|\
awk '{print "d:"$1}' > $REPOFILE
files="$(find . -type f -printf "%P\n"|$FILTER|$FILTER_LOCAL|$FILTER_OWN)"
if [ -n "$files" ]; then
echo $files|xargs md5sum|awk '{print "f:"$1":"$2}' >> $REPOFILE
fi
cd $OLDPWD
}
case $1 in
-*)
printUsage
;;
*)
generateRepoFile $1
;;
esac

71
httpup-repgen-old Executable file
View File

@ -0,0 +1,71 @@
#!/bin/bash
# httpup-repgen - One way sync from an http server to a local directory
# Copyright 2003 (c) Johannes Winkelmann, jw@tks6.net
# The repo file is place on the server. httpsync downloads it, makes the
# update and afterwards moves it to the REPOCURRENTFILE which keeps track
# of the files which have been checked out. The REPOCURRENTFILE contains
# only file names
REPOFILE=REPO
REPOCURRENTFILE=REPO.CURRENT
VERSION=0.5
function info(){
echo $*
}
function debug() {
return # echo $*
}
function printUsage() {
echo "httpup-repgen $VERSION"
echo " Copyright (c) 2003 Johannes Winkelmann"
echo ""
echo "Usage:"
echo " httpup-repgen [directory]"
exit -1
}
function generateRepoFile() {
dir="."
if [ ! "$1" = "" ]; then
dir=$1
fi
if [ ! -d $dir ]; then
echo "Can't generate repository for '$dir': No such directory"
exit -2
fi
echo "Generating repository for directory '$dir'"
OLDPWD=`pwd`
cd $dir
rm -f $REPOFILE || exit -3
touch $REPOFILE
if [ ! "$HS_IGNORE" = "" ]; then
ignore=$HS_IGNORE
ignore="-and $ignore"
debug "$ignore"
fi
for f in `find . -not -name \. $ignore`; do
f=`echo $f|sed -e 's/^\.\///'`
if [ "$f" == "$REPOFILE" ] || [ "$f" = "$REPOCURRENTFILE" ]; then
continue
elif [ -d $f ]; then
echo "d:$f" >> $REPOFILE
else
md5=`md5sum $f|awk '{print $1}'`
echo "f:$md5:$f" >> $REPOFILE
fi
done
cd $OLDPWD
}
if [ "$1" = "--help" ]; then
printUsage
else
generateRepoFile $1
fi

55
httpup-repgen.8 Normal file
View File

@ -0,0 +1,55 @@
.\" man page for httpup-repgen
.\" Johannes Winkelmann, jw@tks6.net
.\"
.\" .PU
.TH "httpup-repgen" "8" "" "" ""
.SH "NAME"
.LP
httpup-repgen \- generate an repository for httpup
.SH "SYNOPSIS"
.B httpup-repgen [target]
.br
.SH "DESCRIPTION"
httpup-repgen creates a repository for httpup. It reads the
.B ~/.httpup-repgen-ignore
and
.B ./.httpup-repgen-ignore
ignore certain patterns of files. This file will be used in a 'grep -v
-f' command. Check the man page of grep(1) if you're uncertain how to use it.
.LP
httpup-repgen again requires the
.B md5sum
utility in order to generate a repository index. The more, a web
server is needed to publish the collection. Possible problems can
arise when publishing files which are executable: some webservers will
refuse to serve them.
.SH "COMMANDS"
.TP
.B httpup-repgen [target directory]
create a repository index file, either for the
.B target directory
specified or the current working directory if called without
additional arguments The repository will be relative to the target directory
.SH "EXAMPLES"
.TP
.B httpup-repgen /home/www/ports/tks6
create a repository relative the the path mentioned
.TP
.B $ echo '\\\\.tar\\\\.gz$' > ~/.httpup-repgen-ignore
.TP
.B $ httpup-repgen /home/www/ports/tks6
create a repository ignoring files matching *.tar.gz
.SH "AUTHORS"
Johannes Winkelmann <jw@tks6.net>
.SH "SEE ALSO"
md5sum(1)
grep(1)

90
httpup-repgen2 Executable file
View File

@ -0,0 +1,90 @@
#!/usr/bin/perl
# httpup-repgen2 - generate a repo for httpup
# --
# Optimized for low CPU load
#
# Copyright 2003 (c) Johannes Winkelmann, jw@tks6.net
use strict;
my $base = @ARGV[0];
if (! -d $base) {
die "No such directory '$base': $!";
}
$_ = $base;
s/(.*)\/$/\1/;
$base = $_;
### Parsing old REPO file
my %repoPorts = ();
if (-f "$base/REPO") {
# print "Parsing REPO \n";
open(IN, "$base/REPO") || die "Can't open repo file: $!";
while (<IN>) {
s/\n//;
my ($t, $md5, $name) = split(/:/);
if ($t eq "f") {
$repoPorts{$name} = $md5;
# print "$name:$repoPorts{$name}\n";
}
}
close(IN);
}
my ($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,
$atime,$mtime,$ctime,$blksize,$blocks) = stat("$base/REPO");
my %resultPorts = ();
my $md5count = 0;
getFiles($base, "", $mtime);
open(OUT, ">$base/REPO") || die "Can't open repo file: $!";
foreach my $key (sort keys %resultPorts) {
if ("$resultPorts{$key}" eq "0") {
print OUT "d:$key\n";
} else {
print OUT "f:$resultPorts{$key}:$key\n";
}
}
close(OUT);
# print ".: Made $md5count md5sum calls :.\n";
sub getFiles() {
# TODO: check double slashes
my $base = $_[0];
my $offset = $_[1];
my $repoMtime = $_[2];
my $dir = "$base/$offset";
opendir(DIR, $dir);
my @entries = readdir(DIR);
foreach my $d (@entries) {
next if ($d eq "." || $d eq "..");
next if ($d =~ "REPO.*");
if (-f "$dir/$d") {
my ($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,
$atime,$mtime,$ctime,$blksize,$blocks) = stat("$dir/$d");
# print "$offset$d (".$repoPorts{"xgalaga/.footprint"}.")\n";
if (!$repoPorts{"$offset$d"} || $repoMtime < $mtime) {
my $md5sum = `md5sum $dir/$d|awk '{print \$1}'`;
$md5sum =~ s/\n//;
$resultPorts{"$offset$d"} = $md5sum;
++$md5count;
close(FILE);
} else {
$resultPorts{"$offset$d"} = $repoPorts{"$offset$d"};
}
} else {
&getFiles($base, "$offset$d/", $repoMtime);
$resultPorts{"$offset$d"} = 0;
}
}
closedir(DIR);
}

74
httpup.8 Normal file
View File

@ -0,0 +1,74 @@
.\" man page for httpup
.\" Johannes Winkelmann, jw@tks6.net
.\"
.\" .PU
.TH "httpup" "8" "" "" ""
.SH "NAME"
.LP
httpup \- an md5sum based one way synchronisation tool for http file
repositories
.SH "SYNOPSIS"
.B httpup <command> URL target
.SH "DESCRIPTION"
httpup performs a one way synchronisation of files published over
http. It is meant for data which is one changed in one place but used
in different other places, for example a ports system. It does only
update the files which are changed (md5sum like).
.SH "COMMANDS"
.TP
.B httpup sync <URL> <target directory>
synchronize the local
.B target directory
with URL
.TP
.B httpup copy <URL> <target directory>
copy the URL to
.B target directory
.TP
.B httpup list <target directory>
List files under httpup's control
.SH OPTIONS
.B --verify-md5, -m:
Verify the md5sum of downloaded files
.B --repofile=<FILE>, -r <FILE>:
Alternative name for the remote REPO file
.B --encode, -e:
URL encode filenames
.SH "CONFIGURATION"
In order to specify proxy server and proxy authentication information, httpup
looks at /etc/httpup.conf which can contain the following four keys:
proxy_host, proxy_port, proxy_user and proxy_pass. Example:
.IP
.nf
proxy_host http://proxy.domain.net
proxy_port 8080
proxy_user joe
proxy_pass very_secret
.i
.IP
.SH "EXAMPLES"
.TP
.B httpup sync http://myhost/ports/tks6 /usr/ports/tks6
Synchronize local copy in /usr/ports/tks6 with the one on
.B myhost
.SH "AUTHORS"
Johannes Winkelmann <jw@tks6.net>

6
httpup.conf.example Normal file
View File

@ -0,0 +1,6 @@
# proxy
proxy_host http://test.proxy.ch
proxy_port 80
proxy_user winkj
proxy_pass very_secret

490
httpup.cpp Normal file
View File

@ -0,0 +1,490 @@
////////////////////////////////////////////////////////////////////////
// FILE: httpup.cpp
// AUTHOR: Johannes Winkelmann, jw@tks6.net
// COPYRIGHT: (c) 2002-2005 by Johannes Winkelmann
// ---------------------------------------------------------------------
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
////////////////////////////////////////////////////////////////////////
#include <iostream>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <dirent.h>
#include "fileutils.h"
#include "httpup.h"
#include "configparser.h"
using namespace std;
const string HttpUp::DEFAULT_REPOFILE = "REPO";
const string HttpUp::REPOCURRENTFILEOLD = "REPO.CURRENT";
const string HttpUp::REPOCURRENTFILE = ".httpup-repo.current";
const string HttpUp::URLINFO = ".httpup-urlinfo";
HttpUp::HttpUp(const HttpupArgparser& argParser,
const string& url, const string& target,
const string& fragment, const string& repoFile,
bool verifyMd5)
: m_baseDirectory(target),
m_remoteUrl(url),
m_fragment(fragment),
m_argParser(argParser),
m_verifyMd5(verifyMd5)
{
if (repoFile != "") {
m_repoFile = repoFile;
} else {
m_repoFile = DEFAULT_REPOFILE;
}
}
int HttpUp::parseCurrent()
{
FILE* fp = fopen((m_baseDirectory+"/"+REPOCURRENTFILE).c_str(), "r");
if (!fp) {
// TODO: remove in 0.3.1
fp = fopen((m_baseDirectory+"/"+REPOCURRENTFILEOLD).c_str(), "r");
if (!fp) {
return -1;
}
}
char input[512];
while (fgets(input, 512, fp)) {
input[strlen(input)-1] = '\0';
m_actions[string(input)] = REMOVE;
}
return 0;
}
int HttpUp::findDiff()
{
FILE* fp = fopen((m_baseDirectory + m_repoFile).c_str(), "r");
if (!fp) {
cerr << "Couldn't open " << m_repoFile << endl;
return -1;
}
char input[512];
struct stat info;
string fileToStat;
while (fgets(input, 512, fp)) {
input[strlen(input)-1] = '\0';
if (input[0] == 'd') {
string dir = input+2;
if (m_fragment != "" &&
dir.substr(0, m_fragment.length()) != m_fragment) {
// doesn't start with fragment
continue;
}
if (m_fragment == dir) {
continue;
}
if (m_fragment != "") {
if (dir.substr(0, m_fragment.length()) == m_fragment &&
dir.length() > m_fragment.length()+1 &&
dir[m_fragment.length()] == '/') {
// strip; matching but hierarchy
dir = dir.substr(m_fragment.length()+1);
if (dir.length() == 0) {
continue;
}
} else {
// strip: fragment is only a substring of dir
continue;
}
}
m_remoteFiles.push_back(dir);
fileToStat = m_baseDirectory + (dir);
if (stat(fileToStat.c_str(), &info) == 0) {
// dir exists
if (!S_ISDIR(info.st_mode)) {
m_actions[dir] = REPLACE_FILE_WITH_DIR;
} else {
m_actions[dir] = NOP;
}
} else {
m_actions[dir] = DIR_CREATE;
}
} else {
int fileNameOffset = 2 + 32 + 1;
// 0+2+32+1 means
// +2 skip the "f:" string
// +32 skip the md5 string
// +1 skip the separator (':') between fileName and md5
string file = input+fileNameOffset;
if (m_fragment != "" &&
file.substr(0, m_fragment.length()) != m_fragment) {
// doesn't start with fragment
continue;
}
if (m_fragment != "") {
if (file.substr(0, m_fragment.length()) == m_fragment &&
file.length() > m_fragment.length()+1 &&
file[m_fragment.length()] == '/') {
file = file.substr(m_fragment.length()+1);
} else {
// skip; fragment is only a substring
continue;
}
}
m_remoteFiles.push_back(file);
fileToStat = m_baseDirectory + (file);
if (stat(fileToStat.c_str(), &info) == 0) {
if (S_ISDIR(info.st_mode)) {
m_actions[file] = REPLACE_DIR_WITH_FILE;
} else {
// file exists
unsigned char result[16];
bool diff = false;
if (FileUtils::fmd5sum(fileToStat, result)) {
input[2+32] = '\0';
diff = verifyMd5sum(input+2, result);
}
if (diff) {
m_actions[file] = FILE_GET;
} else {
m_actions[file] = NOP;
}
}
} else {
m_actions[file] = NEW_FILE_GET;
}
if (m_verifyMd5) {
m_md5sums[file] = string(input+2);
}
}
}
fclose(fp);
return 0;
}
bool HttpUp::verifyMd5sum(const char* input, unsigned char result[16])
{
static char hexNumbers[] = {'0','1','2','3','4','5','6','7',
'8','9','a','b','c','d','e','f'};
bool diff = false;
unsigned char high, low;
for (int i = 0; i < 16; ++i) {
high = (result[i] & 0xF0) >> 4;
low = result[i] & 0xF;
if (*(input+2*i) - hexNumbers[high] ||
*(input+2*i+1) - hexNumbers[low]) {
diff = true;
break;
}
}
return diff;
}
int HttpUp::exec(ExecType type)
{
struct stat info;
if (stat(m_baseDirectory.c_str(), &info)) {
if (FileUtils::mktree(m_baseDirectory.c_str())) {
cerr << "Failed to create base directory "
<< m_baseDirectory << endl;
return -1;
}
}
Config config;
ConfigParser::parseConfig("/etc/httpup.conf", config);
// TODO: check return values.
CURL *curl;
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
char errorBuffer[CURL_ERROR_SIZE];
curl_easy_setopt(curl, CURLOPT_ERRORBUFFER, errorBuffer);
curl_easy_setopt(curl, CURLOPT_FAILONERROR, 1);
curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1);
curl_easy_setopt(curl, CURLOPT_TIMEOUT, 30);
// proxy, proxy auth
if (config.proxyHost != "") {
curl_easy_setopt(curl, CURLOPT_PROXY, config.proxyHost.c_str());
}
if (config.proxyPort != "") {
long port = atol(config.proxyPort.c_str());
curl_easy_setopt(curl, CURLOPT_PROXYPORT, port);
}
string usrpwd;
if (config.proxyUser != "" || config.proxyPassword != "") {
usrpwd = config.proxyUser + ":" + config.proxyPassword;
curl_easy_setopt(curl, CURLOPT_PROXYUSERPWD, usrpwd.c_str());
}
#if 0
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);
#endif
if (!curl) {
cerr << "Failed to initialize CURL engine" << endl;
return -1;
}
cout << "Connecting to " << m_remoteUrl << endl;
int ret = syncOrReturn(curl, errorBuffer);
curl_easy_cleanup(curl);
if (ret == 0) {
if (type == TYPE_SYNC) {
saveRepoCurrent();
} else if (type == TYPE_COPY){
unlink((m_baseDirectory+m_repoFile).c_str());
}
if (type == TYPE_SYNC) {
FILE* fp = fopen((m_baseDirectory+"/"+URLINFO).c_str(), "w");
if (fp) {
fprintf(fp, "%s#%s",
m_remoteUrl.c_str(), m_fragment.c_str());
fclose(fp);
} else {
cerr << "Failed to store urlinfo" << endl;
}
}
}
return ret;
}
int HttpUp::syncOrReturn(CURL* curl, char* curlErrorBuffer)
{
if (getRemoteRepoFile(curl, curlErrorBuffer) != 0) {
cerr << "Failed to get remote repo file" << endl;
return -1;
}
string collectionName =
basename((m_baseDirectory.substr(0, m_baseDirectory.length()-1)).
c_str());
cout << "Updating collection " << collectionName << endl;
// compare with local directory
if (parseCurrent() != 0) {
// -- also "fails" the first time...
// cerr << "Failed to parse local directory" << endl;
// return -1;
}
if (findDiff() != 0) {
cerr << "Failed to check for differences" << endl;
return -1;
}
#if 0
if (m_actions.size() == 0) {
cerr << "No matches found for fragment " << m_fragment << endl;
return -1;
}
#endif
return getChangedFiles(collectionName, curl, curlErrorBuffer);;
}
void HttpUp::saveRepoCurrent()
{
// save current
FILE* current = fopen((m_baseDirectory + REPOCURRENTFILE).c_str(), "w");
if (!current) {
cerr << "Couldn't open "
<< m_baseDirectory << REPOCURRENTFILE << " for writing" << endl;
} else {
list<string>::iterator cit = m_remoteFiles.begin();
for (; cit != m_remoteFiles.end(); ++cit) {
fprintf(current, "%s\n", cit->c_str());
}
fclose(current);
}
// TODO: remove in 0.3.1
FILE* fp = fopen((m_baseDirectory+REPOCURRENTFILEOLD).c_str(), "r");
if (fp) {
fclose(fp);
unlink((m_baseDirectory+REPOCURRENTFILEOLD).c_str());
}
unlink((m_baseDirectory+m_repoFile).c_str());
cout << "Finished successfully" << endl;
}
int HttpUp::getChangedFiles(const string& collectionName, CURL* curl,
char* curlErrorBuffer)
{
int errors = 0;
string fragment = m_fragment;
if (fragment != "") {
fragment += "/";
}
// synchronize
map<string, Action>::iterator it = m_actions.begin();
for (; it != m_actions.end(); ++it) {
if (it->first.substr(0, 3) == "../" ||
it->first.find("/../") != string::npos) {
cerr << " WARNING: Malicious path in remote REPO file: "
<< it->first << endl;
continue;
}
if (it->second == DIR_CREATE) {
cout << " Checkout: "
<< collectionName << "/" << it->first << endl;
mkdir((m_baseDirectory+it->first).c_str(), 0755);
} else if (it->second == NEW_FILE_GET || it->second == FILE_GET) {
if (it->second == NEW_FILE_GET) {
cout << " Checkout: "
<< collectionName << "/" << it->first << endl;
} else if (it->second == FILE_GET) {
cout << " Edit: "
<< collectionName << "/" << it->first << endl;
}
string fileName = it->first;
if (m_argParser.isSet(HttpupArgparser::OPT_ENCODE)) {
char* p = curl_escape(fileName.c_str(), fileName.length());
fileName = p;
curl_free(p);
}
string fileURL = m_remoteUrl+fragment+fileName;
curl_easy_setopt(curl, CURLOPT_URL, fileURL.c_str());
FILE* dlFile = fopen((m_baseDirectory+it->first).c_str(), "w");
if (!dlFile) {
cout << " Failed to open " << it->first
<< " for writing" <<endl;
} else {
curl_easy_setopt(curl, CURLOPT_FILE, dlFile);
CURLcode res = curl_easy_perform(curl);
if (res) {
cout << "Failed to download " << fileURL
<< ": " << curlErrorBuffer << endl;
}
fclose(dlFile);
}
if (m_verifyMd5) {
unsigned char result[16];
if (FileUtils::fmd5sum(m_baseDirectory+it->first, result)) {
bool diff =
verifyMd5sum(m_md5sums[it->first.c_str()].c_str(),
result);
if (diff) {
cerr << "Bad md5sum after download for "
<< it->first << endl;
++errors;
}
} else {
++errors;
}
}
} else if (it->second == REPLACE_DIR_WITH_FILE) {
cout << " Cowardly refusing to overwrite directory '"
<< m_baseDirectory+it->first
<< "' with a file" << endl;
continue;
} else if (it->second == REPLACE_FILE_WITH_DIR) {
cout << " Remove: "
<< collectionName << "/" << it->first
<< " (file)"
<< endl;
int ret = FileUtils::deltree((m_baseDirectory+it->first).c_str());
if (ret == 0) {
cout << " Checkout: "
<< collectionName << "/" << it->first << endl;
mkdir((m_baseDirectory+it->first).c_str(), 0755);
}
} else if (it->second == REMOVE) {
cout << " Delete: "
<< collectionName << "/" << it->first << endl;
if (FileUtils::deltree((m_baseDirectory+it->first).c_str())) {
cout << " Failed to remove " << it->first << endl;
m_remoteFiles.push_back(it->first);
}
}
}
return errors;
}
int HttpUp::getRemoteRepoFile(CURL* curl, char* curlErrorBuffer)
{
// download repo
FILE* dlFile = 0;
string fileName = m_repoFile;
if (m_argParser.isSet(HttpupArgparser::OPT_ENCODE)) {
char* p = curl_escape(fileName.c_str(), fileName.length());
fileName = p;
curl_free(p);
}
string repoURL = m_remoteUrl + fileName;
curl_easy_setopt(curl, CURLOPT_URL, repoURL.c_str());
dlFile = fopen((m_baseDirectory+m_repoFile).c_str(), "w");
if (!dlFile) {
cout << " Failed to open " << m_repoFile << " for writing" << endl;
} else {
curl_easy_setopt(curl, CURLOPT_FILE, dlFile);
CURLcode res = curl_easy_perform(curl);
if (res) {
cerr << " Failed to download " << m_repoFile
<< ": " << curlErrorBuffer << endl;
return -1;
}
fclose(dlFile);
}
return 0;
}

82
httpup.h Normal file
View File

@ -0,0 +1,82 @@
////////////////////////////////////////////////////////////////////////
// FILE: httpup.h
// AUTHOR: Johannes Winkelmann, jw@tks6.net
// COPYRIGHT: (c) 2002-2005 by Johannes Winkelmann
// ---------------------------------------------------------------------
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
////////////////////////////////////////////////////////////////////////
#ifndef _HTTPUP_H_
#define _HTTPUP_H_
#include <string>
#include <list>
#include <map>
#include <curl/curl.h>
#include "httpupargparser.h"
class HttpUp
{
public:
enum Action { NOP,
DIR_CREATE,
REPLACE_FILE_WITH_DIR,
FILE_GET,
NEW_FILE_GET,
REPLACE_DIR_WITH_FILE,
REMOVE };
enum ExecType {
TYPE_SYNC,
TYPE_COPY,
TYPE_MIRROR
};
HttpUp(const HttpupArgparser& argParser,
const std::string& url,
const std::string& target,
const std::string& fragment,
const std::string& repoFile,
bool verifyMd5);
int exec(ExecType execType);
static const std::string DEFAULT_REPOFILE;
static const std::string REPOCURRENTFILE;
static const std::string REPOCURRENTFILEOLD;
static const std::string URLINFO;
private:
int syncOrReturn(CURL* curl, char* curlErrorBuffer);
int getRemoteRepoFile(CURL* curl, char* curlErrorBuffer);
int getChangedFiles(const std::string& collectionName,
CURL* curl, char* curlErrorBuffer);
void saveRepoCurrent();
int findDiff();
int parseCurrent();
static bool verifyMd5sum(const char* input, unsigned char result[16]);
std::map<std::string, Action> m_actions;
std::map<std::string, std::string> m_md5sums;
std::list<std::string> m_remoteFiles;
const std::string m_baseDirectory;
const std::string m_remoteUrl;
const std::string m_fragment;
std::string m_repoFile;
const HttpupArgparser& m_argParser;
bool m_verifyMd5;
};
#endif /* _HTTPUP_H_ */

63
httpupargparser.cpp Normal file
View File

@ -0,0 +1,63 @@
////////////////////////////////////////////////////////////////////////
// FILE: httpupargparser.cpp
// AUTHOR: Johannes Winkelmann, jw@tks6.net
// COPYRIGHT: (c) 2005 by Johannes Winkelmann
// ---------------------------------------------------------------------
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
////////////////////////////////////////////////////////////////////////
#include "httpupargparser.h"
ArgParser::APCmd HttpupArgparser::CMD_SYNC;
ArgParser::APCmd HttpupArgparser::CMD_COPY;
ArgParser::APCmd HttpupArgparser::CMD_LIST;
ArgParser::APOpt HttpupArgparser::OPT_REPOFILE;
ArgParser::APOpt HttpupArgparser::OPT_ENCODE;
ArgParser::APOpt HttpupArgparser::OPT_VERIFY_MD5;
HttpupArgparser::HttpupArgparser()
{
// - sync
addCommand(CMD_SYNC, "sync",
"syncronize local copy with remote repository",
ArgParser::MAX, 2, "[url] [target dir]");
OPT_REPOFILE.init("repofile",
'r',
"alternative name for REPO file",
true,
"NAME");
OPT_ENCODE.init("encode",
'e',
"encode special chars in URL");
OPT_VERIFY_MD5.init("verify-md5",
'm',
"verify md5sum of downloaded files");
addOption(CMD_SYNC, OPT_REPOFILE, false);
addOption(CMD_SYNC, OPT_ENCODE, false);
addOption(CMD_SYNC, OPT_VERIFY_MD5, false);
// - copy
addCommand(CMD_COPY, "copy",
"copy a remote repository to a local directory",
ArgParser::EQ, 2, "<url> <target dir>");
addOption(CMD_COPY, OPT_REPOFILE, false);
addOption(CMD_COPY, OPT_ENCODE, false);
addOption(CMD_SYNC, OPT_VERIFY_MD5, false);
// - list
addCommand(CMD_LIST, "list",
"list files in <directory> which are controlled by httpup",
ArgParser::MAX, 1, "<directory>");
addOption(CMD_LIST, OPT_REPOFILE, false);
}

42
httpupargparser.h Normal file
View File

@ -0,0 +1,42 @@
////////////////////////////////////////////////////////////////////////
// FILE: httpupargparser.h
// AUTHOR: Johannes Winkelmann, jw@tks6.net
// COPYRIGHT: (c) 2005 by Johannes Winkelmann
// ---------------------------------------------------------------------
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
////////////////////////////////////////////////////////////////////////
#ifndef _HTTPUPARGPARSER_H_
#define _HTTPUPARGPARSER_H_
#include "argparser.h"
class HttpupArgparser
: public ArgParser
{
public:
HttpupArgparser();
virtual ~HttpupArgparser() {}
static ArgParser::APCmd CMD_SYNC;
// static ArgParser::APCmd CMD_MIRROR;
static ArgParser::APCmd CMD_COPY;
static ArgParser::APCmd CMD_LIST;
static ArgParser::APOpt OPT_REPOFILE;
static ArgParser::APOpt OPT_ENCODE;
static ArgParser::APOpt OPT_VERIFY_MD5;
std::string getAppIdentification() const
{ return std::string("httpup ") + MF_VERSION + "\n"; }
};
#endif /* _HTTPUPARGPARSER_H_ */

123
main.cpp Normal file
View File

@ -0,0 +1,123 @@
////////////////////////////////////////////////////////////////////////
// FILE: main.cpp
// AUTHOR: Johannes Winkelmann, jw@tks6.net
// COPYRIGHT: (c) 2002-2005 by Johannes Winkelmann
// ---------------------------------------------------------------------
// This program is free software; you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation; either version 2 of the License, or
// (at your option) any later version.
////////////////////////////////////////////////////////////////////////
#include <iostream>
#include <string>
using namespace std;
#include "httpup.h"
#include "fileutils.h"
#include "httpupargparser.h"
int main(int argc, char** argv)
{
HttpupArgparser htap;
htap.parse(argc, argv);
HttpUp::ExecType execType = HttpUp::TYPE_SYNC;
if (htap.command() == HttpupArgparser::CMD_SYNC) {
execType = HttpUp::TYPE_SYNC;
} else if (htap.command() == HttpupArgparser::CMD_COPY) {
execType = HttpUp::TYPE_COPY;
} else if (htap.command() == HttpupArgparser::CMD_LIST) {
string dir = ".";
if (htap.otherArguments().size() > 0) {
dir = htap.otherArguments()[0];
}
FileUtils::listFiles(dir);
exit(0);
} else {
cerr << "Supported commands so far:\n"
<< " sync [<url#fragment>] [<target dir>]\n"
<< " list [<target dir>]\n"
<< "\n"
<< " if target dir is omitted, the "
<< " current working directory is used.\n"
<< " if url is omitted, it is read from the .httpup-urlinfo file"
<< endl;
exit(-1);
}
string url = "";
string fragment = "";
if (htap.otherArguments().size() > 0) {
url = htap.otherArguments()[0];
} else {
FILE* fp = fopen(HttpUp::URLINFO.c_str(), "r");
if (!fp) {
cerr << "Couldn't find " << HttpUp::URLINFO
<< " in current working directory. "
<< endl;
exit(-1);
}
char urlBuf[512];
fgets(urlBuf, 512, fp);
url = urlBuf;
}
if (!htap.isSet(HttpupArgparser::OPT_ENCODE)) {
string::size_type pos = url.find("#");
if (pos != string::npos) {
fragment = url.substr(pos+1);
url = url.substr(0, pos);
}
if (fragment[fragment.size()-1] == '/') {
fragment = fragment.substr(0, fragment.length()-1);
}
}
if (url[url.size()-1] != '/') {
url += '/';
}
string target = "";
if (htap.otherArguments().size() > 1) {
target = htap.otherArguments()[1];
} else {
char* pwd = new char[256];
if (getcwd(pwd, 265) == NULL) {
delete pwd;
pwd = new char[1024];
if (getcwd(pwd, 1024) == NULL) {
cerr << "Path longer then 1024 characters; exiting" << endl;
exit(-1);
}
}
target = pwd;
delete pwd;
}
if (target[target.size()-1] != '/') {
target += '/';
}
string repoFile = "";
if (htap.isSet(HttpupArgparser::OPT_REPOFILE)) {
repoFile = htap.getOptionValue(HttpupArgparser::OPT_REPOFILE);
}
bool verifyMd5 = false;
if (htap.isSet(HttpupArgparser::OPT_VERIFY_MD5)) {
verifyMd5 = true;
}
#if 0
cout << "Sync "
<< (fragment==""?"all":fragment) << " from "
<< url << " to "
<< target << endl;
#endif
HttpUp httpup(htap, url, target, fragment, repoFile, verifyMd5);
return httpup.exec(execType);
}

317
md5.cpp Normal file
View File

@ -0,0 +1,317 @@
/*
* RFC 1321 compliant MD5 implementation,
* by Christophe Devine <devine@cr0.net>;
* this program is licensed under the GPL.
*/
#include <string.h>
#include "md5.h"
#define GET_UINT32(n,b,i) \
{ \
(n) = (uint32) ((uint8 *) b)[(i)] \
| (((uint32) ((uint8 *) b)[(i)+1]) << 8) \
| (((uint32) ((uint8 *) b)[(i)+2]) << 16) \
| (((uint32) ((uint8 *) b)[(i)+3]) << 24); \
}
#define PUT_UINT32(n,b,i) \
{ \
(((uint8 *) b)[(i)] ) = (uint8) (((n) ) & 0xFF); \
(((uint8 *) b)[(i)+1]) = (uint8) (((n) >> 8) & 0xFF); \
(((uint8 *) b)[(i)+2]) = (uint8) (((n) >> 16) & 0xFF); \
(((uint8 *) b)[(i)+3]) = (uint8) (((n) >> 24) & 0xFF); \
}
void md5_starts( struct md5_context *ctx )
{
ctx->total[0] = 0;
ctx->total[1] = 0;
ctx->state[0] = 0x67452301;
ctx->state[1] = 0xEFCDAB89;
ctx->state[2] = 0x98BADCFE;
ctx->state[3] = 0x10325476;
}
void md5_process( struct md5_context *ctx, uint8 data[64] )
{
uint32 A, B, C, D, X[16];
GET_UINT32( X[0], data, 0 );
GET_UINT32( X[1], data, 4 );
GET_UINT32( X[2], data, 8 );
GET_UINT32( X[3], data, 12 );
GET_UINT32( X[4], data, 16 );
GET_UINT32( X[5], data, 20 );
GET_UINT32( X[6], data, 24 );
GET_UINT32( X[7], data, 28 );
GET_UINT32( X[8], data, 32 );
GET_UINT32( X[9], data, 36 );
GET_UINT32( X[10], data, 40 );
GET_UINT32( X[11], data, 44 );
GET_UINT32( X[12], data, 48 );
GET_UINT32( X[13], data, 52 );
GET_UINT32( X[14], data, 56 );
GET_UINT32( X[15], data, 60 );
#define S(x,n) ((x << n) | ((x & 0xFFFFFFFF) >> (32 - n)))
#define P(a,b,c,d,k,s,t) \
{ \
a += F(b,c,d) + X[k] + t; a = S(a,s) + b; \
}
A = ctx->state[0];
B = ctx->state[1];
C = ctx->state[2];
D = ctx->state[3];
#define F(x,y,z) (z ^ (x & (y ^ z)))
P( A, B, C, D, 0, 7, 0xD76AA478 );
P( D, A, B, C, 1, 12, 0xE8C7B756 );
P( C, D, A, B, 2, 17, 0x242070DB );
P( B, C, D, A, 3, 22, 0xC1BDCEEE );
P( A, B, C, D, 4, 7, 0xF57C0FAF );
P( D, A, B, C, 5, 12, 0x4787C62A );
P( C, D, A, B, 6, 17, 0xA8304613 );
P( B, C, D, A, 7, 22, 0xFD469501 );
P( A, B, C, D, 8, 7, 0x698098D8 );
P( D, A, B, C, 9, 12, 0x8B44F7AF );
P( C, D, A, B, 10, 17, 0xFFFF5BB1 );
P( B, C, D, A, 11, 22, 0x895CD7BE );
P( A, B, C, D, 12, 7, 0x6B901122 );
P( D, A, B, C, 13, 12, 0xFD987193 );
P( C, D, A, B, 14, 17, 0xA679438E );
P( B, C, D, A, 15, 22, 0x49B40821 );
#undef F
#define F(x,y,z) (y ^ (z & (x ^ y)))
P( A, B, C, D, 1, 5, 0xF61E2562 );
P( D, A, B, C, 6, 9, 0xC040B340 );
P( C, D, A, B, 11, 14, 0x265E5A51 );
P( B, C, D, A, 0, 20, 0xE9B6C7AA );
P( A, B, C, D, 5, 5, 0xD62F105D );
P( D, A, B, C, 10, 9, 0x02441453 );
P( C, D, A, B, 15, 14, 0xD8A1E681 );
P( B, C, D, A, 4, 20, 0xE7D3FBC8 );
P( A, B, C, D, 9, 5, 0x21E1CDE6 );
P( D, A, B, C, 14, 9, 0xC33707D6 );
P( C, D, A, B, 3, 14, 0xF4D50D87 );
P( B, C, D, A, 8, 20, 0x455A14ED );
P( A, B, C, D, 13, 5, 0xA9E3E905 );
P( D, A, B, C, 2, 9, 0xFCEFA3F8 );
P( C, D, A, B, 7, 14, 0x676F02D9 );
P( B, C, D, A, 12, 20, 0x8D2A4C8A );
#undef F
#define F(x,y,z) (x ^ y ^ z)
P( A, B, C, D, 5, 4, 0xFFFA3942 );
P( D, A, B, C, 8, 11, 0x8771F681 );
P( C, D, A, B, 11, 16, 0x6D9D6122 );
P( B, C, D, A, 14, 23, 0xFDE5380C );
P( A, B, C, D, 1, 4, 0xA4BEEA44 );
P( D, A, B, C, 4, 11, 0x4BDECFA9 );
P( C, D, A, B, 7, 16, 0xF6BB4B60 );
P( B, C, D, A, 10, 23, 0xBEBFBC70 );
P( A, B, C, D, 13, 4, 0x289B7EC6 );
P( D, A, B, C, 0, 11, 0xEAA127FA );
P( C, D, A, B, 3, 16, 0xD4EF3085 );
P( B, C, D, A, 6, 23, 0x04881D05 );
P( A, B, C, D, 9, 4, 0xD9D4D039 );
P( D, A, B, C, 12, 11, 0xE6DB99E5 );
P( C, D, A, B, 15, 16, 0x1FA27CF8 );
P( B, C, D, A, 2, 23, 0xC4AC5665 );
#undef F
#define F(x,y,z) (y ^ (x | ~z))
P( A, B, C, D, 0, 6, 0xF4292244 );
P( D, A, B, C, 7, 10, 0x432AFF97 );
P( C, D, A, B, 14, 15, 0xAB9423A7 );
P( B, C, D, A, 5, 21, 0xFC93A039 );
P( A, B, C, D, 12, 6, 0x655B59C3 );
P( D, A, B, C, 3, 10, 0x8F0CCC92 );
P( C, D, A, B, 10, 15, 0xFFEFF47D );
P( B, C, D, A, 1, 21, 0x85845DD1 );
P( A, B, C, D, 8, 6, 0x6FA87E4F );
P( D, A, B, C, 15, 10, 0xFE2CE6E0 );
P( C, D, A, B, 6, 15, 0xA3014314 );
P( B, C, D, A, 13, 21, 0x4E0811A1 );
P( A, B, C, D, 4, 6, 0xF7537E82 );
P( D, A, B, C, 11, 10, 0xBD3AF235 );
P( C, D, A, B, 2, 15, 0x2AD7D2BB );
P( B, C, D, A, 9, 21, 0xEB86D391 );
#undef F
ctx->state[0] += A;
ctx->state[1] += B;
ctx->state[2] += C;
ctx->state[3] += D;
}
void md5_update( struct md5_context *ctx, uint8 *input, uint32 length )
{
uint32 left, fill;
if( ! length ) return;
left = ( ctx->total[0] >> 3 ) & 0x3F;
fill = 64 - left;
ctx->total[0] += length << 3;
ctx->total[1] += length >> 29;
ctx->total[0] &= 0xFFFFFFFF;
ctx->total[1] += ctx->total[0] < ( length << 3 );
if( left && length >= fill )
{
memcpy( (void *) (ctx->buffer + left), (void *) input, fill );
md5_process( ctx, ctx->buffer );
length -= fill;
input += fill;
left = 0;
}
while( length >= 64 )
{
md5_process( ctx, input );
length -= 64;
input += 64;
}
if( length )
{
memcpy( (void *) (ctx->buffer + left), (void *) input, length );
}
}
static uint8 md5_padding[64] =
{
0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
};
void md5_finish( struct md5_context *ctx, uint8 digest[16] )
{
uint32 last, padn;
uint8 msglen[8];
PUT_UINT32( ctx->total[0], msglen, 0 );
PUT_UINT32( ctx->total[1], msglen, 4 );
last = ( ctx->total[0] >> 3 ) & 0x3F;
padn = ( last < 56 ) ? ( 56 - last ) : ( 120 - last );
md5_update( ctx, md5_padding, padn );
md5_update( ctx, msglen, 8 );
PUT_UINT32( ctx->state[0], digest, 0 );
PUT_UINT32( ctx->state[1], digest, 4 );
PUT_UINT32( ctx->state[2], digest, 8 );
PUT_UINT32( ctx->state[3], digest, 12 );
}
#ifdef TEST
#include <stdio.h>
/*
* those are the standard RFC 1321 test vectors
*/
static char *msg[] =
{
"",
"a",
"abc",
"message digest",
"abcdefghijklmnopqrstuvwxyz",
"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789",
"12345678901234567890123456789012345678901234567890123456789012" \
"345678901234567890"
};
static char *val[] =
{
"d41d8cd98f00b204e9800998ecf8427e",
"0cc175b9c0f1b6a831c399e269772661",
"900150983cd24fb0d6963f7d28e17f72",
"f96b697d7cb7938d525a2f31aaf161d0",
"c3fcd3d76192e4007dfb496cca67e13b",
"d174ab98d277d9f5a5611c2c9f419d9f",
"57edf4a22be3c955ac49da2e2107b67a"
};
int main( int argc, char *argv[] )
{
FILE *f;
int i, j;
char output[33];
struct md5_context ctx;
unsigned char md5sum[16], buffer[1000];
if( argc < 2 )
{
for( i = 0; i < 7; i++ )
{
md5_starts( &ctx );
md5_update( &ctx, (uint8 *) msg[i], strlen( msg[i] ) );
md5_finish( &ctx, md5sum );
for( j = 0; j < 16; j++ )
{
sprintf( output + j * 2, "%02x", md5sum[j] );
}
printf( "test %d ", i + 1 );
if( ! memcmp( output, val[i], 32 ) )
{
printf( "passed\n" );
}
else
{
printf( "failed\n" );
return( 1 );
}
}
}
else
{
if( ! ( f = fopen( argv[1], "rb" ) ) )
{
perror( "fopen" );
return( 1 );
}
md5_starts( &ctx );
while( ( i = fread( buffer, 1, sizeof( buffer ), f ) ) > 0 )
{
md5_update( &ctx, buffer, i );
}
md5_finish( &ctx, md5sum );
for( j = 0; j < 16; j++ )
{
printf( "%02x", md5sum[j] );
}
printf( " %s\n", argv[1] );
}
return( 0 );
}
#endif

18
md5.h Normal file
View File

@ -0,0 +1,18 @@
#ifndef _MD5_H
#define _MD5_H
#define uint8 unsigned char
#define uint32 unsigned long int
struct md5_context
{
uint32 total[2];
uint32 state[4];
uint8 buffer[64];
};
void md5_starts( struct md5_context *ctx );
void md5_update( struct md5_context *ctx, uint8 *input, uint32 length );
void md5_finish( struct md5_context *ctx, uint8 digest[16] );
#endif /* md5.h */