Git Product home page Git Product logo

stumpy's Introduction

STUMPy

Mutaku URL Shortener

WHAT IS STUMPY?

STUMPy is a URL shortener written by xiao_haozi (Mutaku) using the django python web framework.

WHY STUMPY?

There are many url shorteners out there, and STUMPy does not do anything groundbreaking.
However, there are several benefits that encouraged it’s development:

  • you keep all the data and can access it at your will
  • you keep all the code and can access/change it at your will
  • simple to use, simple to run, and simple code
  • because of it’s simplicity, it is easy to understand how url shorteners work and some of the possible optimizations
  • uses the django framework which allows for easy expansion, management, and tweaking
  • django also allows for a nice web UI for administration of all of the data

REQUIREMENTS:

{requirement [link for source]}

INSTALLATION, SETUP, AND TESTING:

See: Django Install
Once you have django installed and running, you can checkout the repo and test things out in a few steps.

  1. Grab the latest release with git.
    git clone https://[email protected]/mutaku/Stumpy.git
  2. Setup a mysql database that stumpy can access.
    create database stumpy;grant all on stumpy.* to ‘SOMEUSER@localhost’ identified by “SOMEPASS”;flush privilieges;
  3. Copy the local_settings.py.dist file to local_settings.py .
    cp local_settings.py.dist local_settings.py
  4. If you do not have a SECRET_KEY generated in Django you will need to generate one by running the keygen script and copy that key into your local_settings.py file in the next step.
    python gimme_key.py
  5. Edit the local_settings.py file and add in your appropriate database variables and your secret key you already had from django or the one you generated above.
  6. Sync the database to the models.
    python manage.py syncdb
  7. Fire up the server and navigate to /admin in your browser. Go to sites→sites and edit the domain name to match the domain you are using. This will initially be set to ‘example.com’.
    python manage.py runserver

PRODUCTION RUNNING:

See: Deploying Django
Once you have your webserver setup to serve the static files, you can set DEBUG=False. Without doing so, static files will not be served up by Django in production mode (DEBUG=False).
Additionally, ensure that you have set your proper FQDN through the admin interface as mentioned in the last step of the setup section.

I have included an example fastcgi script (stumpy.fcgi) that you may point to with your webserver to run Stumpy in a production setting.
The following is an example of a stanza that you could use with Lighttpd to serve up Stumpy.

$HTTP["host"] =~ "example.com" {
  server.document-root = "/home/JohnDoe/Stumpy/"
  fastcgi.server = ( ".fcgi" =>
                     ( "localhost" => (
             "socket" => "/var/lib/lighttpd/stumpy-fastcgi.socket",
                         "bin-path" => "/home/JohnDoe/Stumpy/stumpy.fcgi",
                         "check-local" => "disable",
                         "min-procs" => 2,
             "max-procs" => 4,
                       )
                     ),
                   )
  alias.url = ( "/static/admin"  => "/home/JohnDoe/django/contrib/admin/media/" )
  url.rewrite-once = ( "^(/static/.*)$" => "$1",
                       "^/favicon\.ico$" => "/static/favicon.ico",
               "^(/.*)$" => "/stumpy.fcgi$1" )
}

USAGE:

You will probably be able to see from the urls.py how to go about using Stumpy. Here’s the rundown:

  • [site]/ → index view (recent stumps, popular stumps, and the bookmarklet)
  • [site]/[shorturl] → grab shorturl and redirect
  • [site]/register → register new user [for users – admin can add through admin interface]
  • [site]/admin *→ admin interface
  • [site]/url/?l=[longurl] *→ shorten url [this is how the bookmarklet does it] [needs to be escaped if not using bookmarklet]

*→ These views require authenticated_user and uses sessions

NOTES:

A bookmarklet is ready to use and can be found by visiting the index page (“/”) and dragging the ‘stumpy it!’ link to your bookmarks bar.

To add additional users that may submit URLs, add the user via the admin interface. They do not need to have any admin interface abilities to simply login and and submit URLs.
If you want a user to have permission to edit and delete urls (why?) you would have to enable those permissions in the user management in the admin interface.

FAQ:

[coming soon]

HELP:

Most of the information you might need can be found in the Django docs here: Django
For Stumpy specific help, see Stumpy

stumpy's People

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

z-pulley

stumpy's Issues

weird stripping in request data (http://google.com -> http:/google.com)

urls that are encoded get passed to /ur/encoded_url
the encoded_url always yeilds:
http://google.com -> http:/google.com

// -> / for some reason.
admin interface submission is fine (right to model) something is just off with lighttpd+request parsing !?

djang-user google group link: http://groups.google.com/group/django-users/browse_thread/thread/76919b39159b66db

could manually look for this and adjust but that's kinda meh and doesn't solve the issue:

stump_split = list(this_stump.partition(":")) if stump_split[1] and not stump_split[2].startswith("//"): stump_split[2] = "/"+stump_split[2] this_stump = ''.join(stump_split)

self stump

Add in a check for shortening links that are already shortened (self domain) using url parsing like in the old stumpy.

hardcoded urls

-> In the index the BOOKMARKLET has hardcoded domain in it.
-> Self stump check in shortener.views.submit has hardcoded domain in it.

something like this should work:

Site.objects.get_current().domain
u'example.com'

but that's obviously not correct.

old stumpy urls

should we dump in the old urls?

all we would have to do us drop the table (when ready), then take the old database and for each row grab the full url and use urllib to do a "/url/FULLURL" kind of thing for each and since the shortened url is made off the auto_incrementing id column for each insert, they should match up.
Once they are verified to match, we can backup old database and take it down.

branches - public vs dev

When the last bugs are finalized, create a public branch to put at the main mutaku github and then keep a private development branch for personal use and development that can be pushed out to the public one and vice versa.

clean urls

add another layer of url cleaning before dumping.
right now it does a verify_exists which is decent i guess.

old way was to run it through an htmlcleaner but lets do something more django built-in.

database errors

create a 404 or other nice error when hitting a dupe entry (now spits out a mysql error on the unique field [hashurl])

admin interface is broken

Can't seem to access the admin interface anymore with urls.py

# show the index from /
url(r'^$', 'shortener.views.index'),
# get a url for redirection /shorty
url(r'^(?P<short>\w+)/$', 'shortener.views.detail'),
# send a url to be shortened from /url/someencodedurl
url(r'^url/(?P<url>\S+)/$', 'shortener.views.submit'),

# Uncomment the admin/doc line below to enable admin documentation:
url(r'^admin/doc/', include('django.contrib.admindocs.urls')),
# Uncomment the next line to enable the admin:
url(r'^admin/', include(admin.site.urls)),

judging by the 500 error it seems to be sending it to the submit view

Environment:

Request Method: GET
Request URL: http://192.168.11.4:8001/admin/

Django Version: 1.3 SVN-16009
Python Version: 2.6.5
Installed Applications:
['django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.admin',
'django.contrib.admindocs',
'shortener']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware')

Traceback:
File "/home/mugen/programming/django/django/core/handlers/base.py" in get_response

  1.                     response = callback(request, _callback_args, *_callback_kwargs)
    
    File "/home/mugen/programming/Stumpy/shortener/views.py" in detail
    __ 11. thisurl = url.objects.get(shorturl=short) __
    File "/home/mugen/programming/django/django/db/models/manager.py" in get
  2.     return self.get_query_set().get(_args, *_kwargs)
    
    File "/home/mugen/programming/django/django/db/models/query.py" in get
  3.                 % self.model._meta.object_name)
    

Exception Type: DoesNotExist at /admin/
Exception Value: url matching query does not exist.

Long urls that are too long

need to trim the full urls... caused a mess when they are shown in full.
shows up in admin interface and in the index, e.g. ...

could use this, at least in the template:
{{ some_list.longurl|slice:":40"}}
this would chop at 40 characters... but we would probably want to do an if {{some_list.longurl|length}} >40 then we add "..." after that template insertion to indicate to the user that we chopped it.

key generation

string.punctuation includes " and ' which could easily mess up copying and including for use as a variable such as SECRET_KEY which be default looks like:
SECRET_KEY = ''

string.punctuation
'!"#$%&'()*+,-./:;<=>?@[]^_`{|}~'

Either replace " and ' with something else or triple quote SECRET_KEY like:
SECRET_KEY = ''' '''

post url returns

Need an informative return after posting url to provide the shortened version.
Kinda does this now but it's ugly.

Show User Stumps on Main Page

Show the currently logged in users stumps on the main page so they don't always have to go to the admin interface to view their personal stumps.

login next url query string issue

query strings in submitted urls are stripped off when submitting and not logged in...
redirected to login page where url is stored as 'next=' so the query string is getting sucked into the url grab then.

bookmarklet size

too small... make a tad taller and double the width maybe.

or adjust the css for the post results html.

admin interface is broken

Can't seem to access the admin interface anymore with urls.py

judging by the 500 error it seems to be sending it to the submit view

Environment:

Request Method: GET
Request URL: http://192.168.11.4:8001/admin/

Django Version: 1.3 SVN-16009
Python Version: 2.6.5
Installed Applications:
['django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.admin',
'django.contrib.admindocs',
'shortener']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware')

Traceback:
File "/home/mugen/programming/django/django/core/handlers/base.py" in get_response

  1.                     response = callback(request, _callback_args, *_callback_kwargs)
    
    File "/home/mugen/programming/Stumpy/shortener/views.py" in detail
    __ 11. thisurl = url.objects.get(shorturl=short) __
    File "/home/mugen/programming/django/django/db/models/manager.py" in get
  2.     return self.get_query_set().get(_args, *_kwargs)
    
    File "/home/mugen/programming/django/django/db/models/query.py" in get
  3.                 % self.model._meta.object_name)
    

Exception Type: DoesNotExist at /admin/
Exception Value: url matching query does not exist.

urls.py regex

right now the way things work is that all strings after / go to decoding into a url EXCEPT if string == admin or account (since these come before the general url decoding bit)

This is okay until someone would hit over 150million links:

import shortener.base62 as s
s.Decode('admin').num
150947331
s.Decode('account').num
579179102503L

not really an urgent issue (of course) but programmatically it sucks.
==> fix regex

unicode issue with url input

Environment:

Request Method: GET
Request URL: http://192.168.11.4:8001/url/http://forecast.weather.gov/MapClick.php?CityName=Philadelphia&state=PA&site=PHI&lat=39.9525&lon=-75.1657/

Django Version: 1.3 SVN-16009
Python Version: 2.6.5
Installed Applications:
['django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.admin',
'django.contrib.admindocs',
'shortener']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware')

Traceback:
File "/home/mugen/programming/django/django/core/handlers/base.py" in get_response

  1.                     response = callback(request, _callback_args, *_callback_kwargs)
    
    File "/home/mugen/programming/Stumpy/shortener/views.py" in submit
  2. thisurl = url(longurl=theurl)

Exception Type: TypeError at /url/http://forecast.weather.gov/MapClick.php?CityName=Philadelphia&state=PA&site=PHI&lat=39.9525&lon=-75.1657/
Exception Value: 'unicode' object is not callable

from shortener.models import url
url = u'http://t04u.be/'
type(url)
<type 'unicode'>
b = url(longurl=url)
Traceback (most recent call last):
File "", line 1, in
TypeError: 'unicode' object is not callable
from django.utils.encoding import smart_str
b = url(longurl=smart_str(url))
Traceback (most recent call last):
File "", line 1, in
TypeError: 'unicode' object is not callable
smart_str(url)
'http://t04u.be/'
type(smart_str(url))
<type 'str'>
a = smart_str(url)
type(a)
<type 'str'>
b = url(longurl=a)
Traceback (most recent call last):
File "", line 1, in
TypeError: 'unicode' object is not callable

dupe submit

right now can't dupe submit as it picks up on validation error at hashurl field.
because debug=True (for now) it spits back validation error.

This should instead give back a detail view with the existing short url.
[do something like try: except ValidationError (duplication): blah]

user auth feature [how to handle registration]

implement the chainsaw (or django session) so that, if wanted, could associated with a specific user jsut for being able to 'show all my stumps'.

however, it could also be done by using real authentication. And so when the user clicks bookmarklet and session has expired, the pop up will spit in the login first then redirect back to submit link. This way, the cookie column would then be just user names that we can show your username associated stumps if you ask for them.

production switching

after setting up lighty,
remove the debug switch so we aren't feeding out all kinds of code to the public.

Double posts

Seems that it's passing nicely over a double post on updates. (editing a past url and changing to an existing) But I think new creation dupes are picked up fine.


updates to the hits column also trigger this duplication problem.

Have to differential the inserts on the urls vs updates for duplication checks.

no protocol specific links break redirect()

redirect from django.shortcuts is used to do a:
return redirect(stump.longurl)

but if longurl == 'google.com', the redirect breaks and shows the error in parsign urls.py?!?

for now i've just added a hack to add a default http:// protocol so google.com becomes http://google.com but this is meh as well.

for now this saves the redirect.

updates and dupes again

seems that i can still create dupes but breaks on creating the short url (after the save method) since it finds multiple returns on longurl.

However, if i scrape through this before saving, i can't update values later like +=1 hits.

admin clickable link

Need a clickable link in admin interface so when sorting or searching links, you can actually click to visit rather than copy and paste out.

Index

We need a nice index template to match the one currently at t04u.be

hash url and admin interface

since the hashurl is currently created via views.py, if you enter a url in teh admin interface, no hashurl is created.

move hashurl creation to model level.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.