Index of /~snl/UpYours
Name Last modified Size Description
![[DIR]](/icons/back.gif)
Parent Directory 10-Dec-2004 23:10 -
![[TXT]](/icons/text.gif)
LATEST_IS_0.1.3 10-Oct-2001 13:27 0k
![[ ]](/icons/compressed.gif)
upyours-0.1.2.tar.gz 10-Oct-2001 01:11 14k
upyours-0.1.3.tar.gz 10-Oct-2001 13:27 15k
upyours-latest.tar.gz 10-Oct-2001 13:27 15k
-*- mode:text; indent-tabs-mode:nil -*-
Time-stamp: <2001-10-10 13:17:56 EDT>
WHAT IS THIS THING
Sometimes, you just need to upload a file and you don't have your
crypto bits handy. In this day and age, people should always use
scp (or pscp.exe) to copy files around, but users are users, and
even clueful people sometimes find themselves in a position where
it's not easy, convenient or even possible to use anything but a
stupid web interface (a kiosk on a conference floor for example).
UpYours! is a simple solution to this problem. The idea is to
provide a way for file uploads to occur that:
(a) doesn't leak information about the server to the (potentially
malicious) client;
(b) does not present anything useful to people who want to use
the server as a drop-point for files;
(c) does not open the server up to DoS attacks;
(d) is simple to use, and can be deployed over http and https;
(e) does not force the client to reveal any secret information
There are three parts to UpYours!
(1) upyours.cgi - the CGI form that presents the client with
a username and a file upload field;
(2) uploads - a program that users can run on the server in
a shell session to see their uploads, retrieve them, delete
them, etc;
(3) upclean - a program meant to be run as a cron job to
periodically flush out old uploads that haven't been claimed
Okay, I lied, there are four parts:
(4) An UpYoursConfig.pm module that you edit with settings for
your server and install; this is optional, you can just edit
the code instead of you want.
Well, actually, there's some installation stuff... Of the several
and various parts of UpYours!, we also include:
(5) Some scripts to install UpYours!, set up the database, and
mung the allowed users table.
(6) Utils.pm, a subset of my bag of perl tricks that I use
for various things, like logging.
You should probably be familiar with SQL databases if you want to
install UpYours!. A passing familiarity with perl couldn't hurt,
either.
PART 1: THE CGI
The CGI accepts a username and a file. The username need not have
anything to do with real usernames on the server (and in fact
probably should not). The form uses DBI to talk to a database on
the server which contains two tables:
uploaders - a table of usernames and their upload quotas in bytes
uploads - a table of files that have been uploaded
Inputs are cleansed of nasty things like poison NUL bytes and
control characters. Additionally, file path names are stripped of
drive and directory information. The form can be configured to
upload with throttling I/O to limit the amount of bandwidth that
can be consumed by file uploads via UpYours!. The form gives no
indication to the user of whether or not the user name is valid,
over quota, etc. As the file is uploaded, an MD5 of its contents
is computed, along with some random bits on the server side that
the client cannot guess (so that the client or a MiTM cannot guess
the MD5, which is only really important in very obscure cases,
but oh well). Each user has a quota in bytes in the database;
the size of the upload plus all of the other uploads pending for
the user are added up and compared to the quota. If this upload
would put the user over quota, or the user does not exist, or there
is already an upload pending with the same size and the same
(source) file name (sans directory and drive), the new upload is
discarded. Otherwise, the upload is stored using the MD5 as the
file name, in a directory that is not accessible to the outside
world. Whether the upload is discarded or stored, the MD5 is
reported to the client as a "ticket #" for the upload. Successful
uploads are recorded in the uploads table as pending and charged
against the user's upload quota until they are retrieved.
PART 2: THE SERVER-SIDE PROGRAM
A legitimate user of the server can ssh in and get a shell session
going; doing this securely is beyond the scope of UpYours!, but we
are assuming here that the server is set up sanely in this regard
to begin with (i.e. no unencrypted access, etc). Once logged in,
the uploads command will show them their pending uploads. These
files may be discarded or retrieved, using either the MD5 or the
name of the file. Doing either thing causes the upload to be
removed from the UpYours! area and the record deleted from the
database.
PART 3: THE CRON JOB
Per-user quotas should be set sanely; 10MB should be plenty for
normal use, but disk is cheap. Eventually, however, you don't want
UpYours! to start eating space ad infinitum, so it is recommended
that a cron job be configured to run the upclean program every so
often. Upclean is given a maximum time as a parameter, and cleans
out UpYours! uploads that have been pending at least that long.
INSTALLATION AND CONFIGURATION: TEN STEPS TO A SLIMMER, SEXIER YOU
(-1) Check the REQUIREMENTS section before proceding, make sure
you have all the required perl modules installed
(0) Set up some part of your web server to allow CGI bins;
also, have MySQL, PostgreSQL or some other DBI-accessible
database running on your server;
(1) Create the upyours database however you do that; for
PostgreSQL it's:
$ createdb upyours
Make sure all of the potential UpYours users have access
to the database. Make sure the user that the Web server
runs as has access to the database.
(1) Copy the UpYoursConfig.pm.in file to UpYoursConfig.pm
and edit it to taste.
(2) Run the upinstall.pl script:
$ perl ./upinstall.pl
This will use DBI to create the right tables and make sure
things look sane. You do not need to install the upinstall.pl
script anywhere.
(3) Put the UpYoursConfig.pm perl module somewhere that perl
programs can find it; somewhere on @INC would be a nice
idea. Say
$ perl -V
to see what directories are on @INC. On many BSD type systems,
some place like /usr/local/lib/perl5/site_perl/5.005/ would be
a good directory to put UpYoursConfig.pm:
$ cp UpYoursConfig.pm /usr/local/lib/perl5/site_perl/5.005
Alternatively, you can put it wherever you like and add a -I
option to all of the perl programs so that they can find it.
If you don't know what I'm talking about, forget it, just do
the @INC thing.
(3.5) Everything I said in #3 for UpYoursConfig.pm goes double for
Utils.pm, which should be put somewhere. This part is not
optional, unless you're a perl hacker, in which case you can
figure out what to do yourself if you're so smart
(4) To add users to UpYours! use the upuser.pl script:
$ perl upuser.pl [username quota]
will add them to the database. If you don't give upuser.pl
(enough) arguments, it will interact with you in a reasonable
way.
(4') Alternatively, you can just use the SQL monitor for your
database package to insert stuff into the uploaders table in
the upyours database. This is what I do, but I thought I'd
be nice and give you a program to do it for you. If you
do this manually in SQL, make sure to also create directories
under $SAVE_DIR/files/ for each user with permissions such
that the web server can write into them (g+rwx probably).
(5) Edit upyours.cgi, uploads and upclean to make sure their first
line points at the right version of perl. Yes, I know if I
wasn't such a lazy bastard and wrote a Makefile.PL that this
would all be taken care of; feel free to contribute one.
(6) Drop upyours.cgi into the CGI bin directory
(7) Drop uploads into some reasonable place like /usr/local/bin
(8) Drop upclean into some reasonable place like /usr/local/bin
(9) Configure a cron job to run upclean every once in a while;
it takes a single argument, which is a maximum age in seconds.
$ upclean 3600
blows away UpYours uploads older than an hour.
(10) Enjoy
REQUIREMENTS
Time::HiRes for usleep, gettimeofday (used by throttling code)
DBI this is all based on databases
DBD::{Pg,mysql,whatever} interface to your database
Pod::Usage my command-line scripts use this for -help
CGI this is a web interface, after all
Digest::MD5 the filenames are md5s
BUGS, PROBLEMS, ODDITIES AND CAVEATS
UpYours! was written with the mindset that the random clients
connecting to your web server are nasty, unhygenic, malicious
and smelly, but legitimate users who have ssh'ed in and have
shell prompts are nice, clean, orderly and trustworthy. I take
no precautions against users abusing the uploads command to
steal other peoples' uploads, etc. If this does not fit with
your situation, then some additional hacking is required to
make things better, but I really think it's a lost cause. Feel
free to contribute code, of course, but I'm not going to worry
about this for my own purposes.
Seriously, PC's are, what, $5 now, including a reasonable NIC?
If you're hosting shell accounts, be careful, and if you can't
be careful, then have fresh, known-good install media and a
supply of coffee handy.
One mildly strange thing upyours.cgi does is change leading dots
in source filenames to "dot.". This is more to avoid surprises
on the part of users ("hey! uploads says it grabbed the file but
I don't see it! WE'VE BEEN HACKED!") than anything else.
The uploads command has some special-purpose code for mysql and
PostgreSQL to deal with the differences in their timestamp formats.
If you want to use something other than one of those two, you
may have to tweak this code (though the fallback will still print
useful, if not pretty results).
Do not taunt happy fun file uploader.
TODO
If you're itching for something to do, feel free to pick one of
these tasks:
+ Make the form more configurable and prettier.
+ Make the output from uploads nicer/smarter (deal with long filenames)
+ The file permissions fu is so dumb; could get around it by storing
the actual files in the database, too, as blobs.
LOCATION
You can always get the most recent version from
http://www.bitsend.com/~snl/UpYours/
CREDITS
Sean Levy <snl@hardbits.com> wrote this one night at 2am. He's
pretty sure there aren't any bugs, but feel free to e-mail him
if there are.
LICENSE
You can do whatever you want with this so long as you
(a) give the original author credit;
(b) give the original author back any changes you make;
(c) don't restrict the original author from doing what he
wants with your changes
I don't know what kind of Open Source license this is, or if
the Open Source mavens would even consider it Open Source, and
I don't particularly give a rat's ass about any of that.
Have a nice damned day.
COPYRIGHT
Copyright (C) 2001 by Sean Levy. All Rights Reserved.
HISTORY
$Id: README,v 1.5 2001/10/10 17:21:23 snl Exp $
0.1.1 09.Oct.2001 snl First release
0.1.2 09.Oct.2001 snl Added throttling; not quite right yet.
Added $MAX_DOWNLOAD
0.1.3 10.Oct.2001 snl (Note to self: change meds)
Added REQUIREMENTS section and more
stuff under ODDITIES in this README
Fixed stupid typos in UpYoursConfig.pm.in
Made parsets() know about at least
mysql vs Pg timestamps