Git Product home page Git Product logo

yourlabs / django-cities-light Goto Github PK

View Code? Open in Web Editor NEW

This project forked from coderholic/django-cities

335.0 24.0 126.0 17.8 MB

A simple app providing three models: Country, Region and City model. Also provided, a command to insert or update data from geonames database dumps. Status: stable.

Home Page: http://django-cities-light.rtfd.org/

License: MIT License

Python 99.71% Dockerfile 0.29%

django-cities-light's Introduction

https://secure.travis-ci.org/yourlabs/django-cities-light.svg?branch=master

django-cities-light -- Simple django-cities alternative

This add-on provides models and commands to import country, subregion, region/state, and city data in your database.

The data is pulled from GeoNames and contains cities, subregions, regions/states and countries.

Spatial query support is not required by this application.

This application is very simple and is useful if you want to make a simple address book for example. If you intend to build a fully featured spatial database, you should use django-cities.

Requirements:

  • Python >= 3.8
  • Django >= 3.2
  • MySQL or PostgreSQL or SQLite.

Yes, for some reason, code that used to work on MySQL (not without pain xD) does not work anymore. So we're now using django.db.transaction.atomic which comes from Django 1.6 just to support MySQL quacks.

Features

  • GraphQL support
  • Built-in admin support
  • Rest-Framework support
  • Ajax Select Lookup support

Upgrade

See CHANGELOG.

Installation

Install django-cities-light:

pip install django-cities-light

Or the development version:

pip install -e [email protected]:yourlabs/django-cities-light.git#egg=cities_light

Add cities_light to your INSTALLED_APPS.

Configure filters to exclude data you don't want, ie.:

CITIES_LIGHT_TRANSLATION_LANGUAGES = ['fr', 'en']
CITIES_LIGHT_INCLUDE_COUNTRIES = ['FR']
CITIES_LIGHT_INCLUDE_CITY_TYPES = ['PPL', 'PPLA', 'PPLA2', 'PPLA3', 'PPLA4', 'PPLC', 'PPLF', 'PPLG', 'PPLL', 'PPLR', 'PPLS', 'STLMT',]

Now, run migrations, it will only create tables for models that are not disabled:

./manage.py migrate

Data import/update

Finally, populate your database with command:

./manage.py cities_light

This command is well documented, consult the help with:

./manage.py help cities_light

By default, update procedure attempts to update all fields, including Country/Region/Subregion/City slugs. But there is an option to keep them intact:

./manage.py cities_light --keep-slugs

Get more cities

The configuration parameter CITIES_LIGHT_CITY_SOURCES, comes with the default value http://download.geonames.org/export/dump/cities15000.zip that has cities with a population over 15000, if you need to load cities with less population please use another source. For the list of available source please check here: http://download.geonames.org/export/dump/readme.txt

Using fixtures

Geonames.org is updated on daily basis and its full import is quite slow, so if you want to import the same data multiple times (for example on different servers) it is convenient to use fixtures with the helper management command:

./manage.py cities_light_fixtures dump
./manage.py cities_light_fixtures load

To reduce space, JSON fixtures are compressed with bzip2 and can be fetched from any HTTP server or local filesystem.

Consult the help with:

./manage.py help cities_light_fixtures

Development

Create development virtualenv (you need to have tox installed in your base system):

tox -e dev
source .tox/dev/bin/activate

To run the test project, with the folder of the project as the current directory, run:

export PYTHONPATH="${PYTHONPATH}:/app/src"
docker run  -d postgres -p 5432:5432

Then run the full import:

test_project/manage.py migrate
test_project/manage.py cities_light

There are several environment variables which affect project settings (like DB_ENGINE and CI), you can find them all in test_project/settings.py.

For example to change the database engine, you can run:

export DB_ENGINE=postgresql
export DB_HOST=192.168.0.118
export DB_NAME=app
export DB_USER=postgres
export DB_PORT=5432

To run the test suite you need to have postgresql or mysql installed with passwordless login, or just use sqlite. Otherwise the tests which try to create/drop database will fail.

Running the full test suite:

tox

To run the tests in specific environment use the following command:

tox -e py37-django31-sqlite

And to run one specific test use this one:

tox -e py37-django31-sqlite -- cities_light/tests/test_form.py::FormTestCase::testCountryFormNameAndContinentAlone

To run it even faster, you can switch to specific tox virtualenv:

source .tox/py37-django18-sqlite/bin/activate
CI=true test_project/manage.py test cities_light.tests.test_form.FormTestCase.testCountryFormNameAndContinentAlone

If you want to build the docs, use the following steps:

source .tox/dev/bin/activate
cd docs
make html

TODOS

Resources

You could subscribe to the mailing list ask questions or just be informed of package updates.

django-cities-light's People

Contributors

adamb70 avatar almalki avatar chrisv2 avatar coderholic avatar dpoirier avatar drivard avatar georgema1982 avatar greenday2 avatar guts avatar jezdez avatar jpic avatar magnunleno avatar marianoeramirez avatar max-arnold avatar mbaragiola avatar mep-um avatar pfouque avatar pidelport avatar predatell avatar qarterd avatar roelio avatar screamndigit avatar shahzad31 avatar shanto avatar smithdc1 avatar st4lk avatar suquant avatar timrchavez avatar wswld avatar xrmx avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

django-cities-light's Issues

Data not populated into database

Strange problem here: I tried to install django-autocomplete-light and see good sql in DEBUG log but my database tables are empty. I tested this sql manually - it works (I may login to PhpPostgreAdmin (as database user, settings.py) and manually execute, for example,

INSERT INTO "cities_light_country" ("name_ascii", "slug", "geoname_id", "alternate_names", "name", "code2", "code3", "continent", "tld") VALUES ('Andorra', 'andorra', 3041565, '', 'Andorra', 'AD', 'AND', 'EU', 'ad') RETURNING "cities_light_country"."id";

)... Also I tried SQLite, (this three tables in Django admin were empty too)

./manage.py cities_light --force-import-all
WARNING Assuming local download is up to date for http://download.geonames.org/export/dump/countryInfo.txt
INFO Importing countryInfo.txt
DEBUG (0.122) SELECT "cities_light_country"."id", "cities_light_country"."name_ascii", "cities_light_country"."slug", "cities_light_country"."geoname_id", "cities_light_country"."alternate_names", "cities_light_country"."name", "cities_light_country"."code2", "cities_light_country"."code3", "cities_light_country"."continent", "cities_light_country"."tld" FROM "cities_light_country" WHERE "cities_light_country"."code2" = 'AD' ; args=('AD',)
DEBUG (0.060) INSERT INTO "cities_light_country" ("name_ascii", "slug", "geoname_id", "alternate_names", "name", "code2", "code3", "continent", "tld") VALUES ('Andorra', 'andorra', 3041565, '', 'Andorra', 'AD', 'AND', 'EU', 'ad') RETURNING "cities_light_country"."id"; args=('Andorra', u'andorra', 3041565, u'', u'Andorra', 'AD', 'AND', 'EU', 'ad')
DEBUG (0.060) SELECT "cities_light_country"."id", "cities_light_country"."name_ascii", "cities_light_country"."slug", "cities_light_country"."geoname_id", "cities_light_country"."alternate_names", "cities_light_country"."name", "cities_light_country"."code2", "cities_light_country"."code3", "cities_light_country"."continent", "cities_light_country"."tld" FROM "cities_light_country" WHERE "cities_light_country"."code2" = 'AE' ; args=('AE',)
DEBUG (0.060) INSERT INTO "cities_light_country" ("name_ascii", "slug", "geoname_id", "alternate_names", "name", "code2", "code3", "continent", "tld") VALUES ('United Arab Emirates', 'united-arab-emirates', 290557, '', 'United Arab Emirates', 'AE', 'ARE', 'AS', 'ae') RETURNING "cities_light_country"."id"; args=('United Arab Emirates', u'united-arab-emirates', 290557, u'', u'United Arab Emirates', 'AE', 'ARE', 'AS', 'ae')
...

Python 2.7.3, Mac OS, virtualenv

pip freeze:

Django==1.4.1
PIL==1.1.7
South==0.7.6
distribute==0.6.28
django-appconf==0.5
django-autoslug==1.6.1
django-cities-light==2.0.4
django-compressor==1.2
django-debug-toolbar==0.9.4
django-easyfilters==0.4
django-profiles==0.2
django-registration==0.8
django-registration-invitationcode-backend==0.0.1
django-taggit==0.9.3
django-taggit-templatetags==0.4.6dev
django-templatetag-sugar==0.1
docutils==0.9.1
progressbar==2.2
psycopg2==2.4.5
py-bcrypt==0.2
python-dateutil==2.1
six==1.2.0
wsgiref==0.1.2

settings.py:

PROJECT_PATH = path.dirname(path.dirname(path.realpath(__file__)))
...
CITIES_LIGHT_TRANSLATION_LANGUAGES = (
    'en',
    'ru',
)

CITIES_LIGHT_DATA_DIR = path.join(PROJECT_PATH, 'data')

How to remove the valid character from the name

On top of Country, region and state name, I can see some invalid character like -.
How to remove that invalid character "-"

Example

Arunāchal Pradesh
Bihār
Chandīgarh
Karnātaka
Mahārāshtra

Error on Django 1.5 with python 2.7 loading initial Data

Hello , when i try to load initial data with ./manage.py cities-light it return this error :
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 42: ordinal not in range(128)

for more infor i paste --traceback return .

Traceback (most recent call last):
File "C:\Python27\lib\site-packages\django\core\management\base.py", line 222, in run_from_argv
self.execute(_args, *_options.dict)
File "C:\Python27\lib\site-packages\django\core\management\base.py", line 255, in execute
output = self.handle(_args, *_options)
File "C:\Python27\lib\site-packages\django\db\transaction.py", line 223, in inner
return func(_args, *_kwargs)
File "C:\Python27\lib\site-packages\cities_light\management\commands\cities_light.py", line 128, in handle
geonames = Geonames(url, force=force)
File "C:\Python27\lib\site-packages\cities_light\geonames.py", line 30, in init
self.downloaded = self.download(url, self.file_path, force)
File "C:\Python27\lib\site-packages\cities_light\geonames.py", line 49, in download
'%a, %d %b %Y %H:%M:%S %Z')
File "C:\Python27\lib_strptime.py", line 467, in _strptime_time
return _strptime(data_string, format)[0]
File "C:\Python27\lib_strptime.py", line 308, in _strptime
format_regex = _TimeRE_cache.compile(format)
File "C:\Python27\lib_strptime.py", line 265, in compile
return re_compile(self.pattern(format), IGNORECASE)
File "C:\Python27\lib_strptime.py", line 259, in pattern
self[format[directive_index]])
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe1 in position 42: ordinal not in range(128)

Not longer being able to load initial data

Hello,

Until yesterday I was being able to import the data using python manage.py cities_light --force-import-all. Now I just started getting the following exception:

I tried un-installing django-cities-light and re-installing, also tried on a two separated systems with the same result.

Any help will be appreciated.

(travelsite)dev@server:~/www/travelsite/travelsite python manage.py cities_light --force-import-all --traceback

No handlers could be found for logger "cities_light"
RAM used: 22724 kB Time: 00:00:00 Done: 100%|################################################################################################################################################################################################################################|
Traceback (most recent call last):
File "/home/dev/envs/travelsite/local/lib/python2.7/site-packages/django/core/management/base.py", line 222, in run_from_argv
self.execute(_args, *_options.dict)
File "/home/dev/envs/travelsite/local/lib/python2.7/site-packages/django/core/management/base.py", line 255, in execute
output = self.handle(_args, *_options)
File "/home/dev/envs/travelsite/local/lib/python2.7/site-packages/django/db/transaction.py", line 223, in inner
return func(_args, *_kwargs)
File "/home/dev/envs/travelsite/local/lib/python2.7/site-packages/cities_light/management/commands/cities_light.py", line 140, in handle
self.region_import(items)
File "/home/dev/envs/travelsite/local/lib/python2.7/site-packages/cities_light/management/commands/cities_light.py", line 223, in region_import
name = items[1]
IndexError: list index out of range

IndexError: list index out of range (2.0.7)

hello,
i'm unable to use cities_light, due to this error:
(django15x)machine:project me$ python manage.py shell
Python 2.7.3 (v2.7.3:70274d53c1dd, Apr 9 2012, 20:52:43)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)

from django.db import models
from cities_light.models import City
class Person(models.Model):
... name = models.CharField(max_length=100)
... location = models.ForeignKey(City, blank = True, null = True)
...
Traceback (most recent call last):
File "", line 1, in
File "/Users/me/python/virtualenv/django15x/lib/python2.7/site-packages/django/db/models/base.py", line 93, in new
kwargs = {"app_label": model_module.name.split('.')[-2]}
IndexError: list index out of range

the same error arises if using django 1.4.2. What am i doing wrong?

thanks in advance for help,

g.

Not working with Django 1.7 news migrations structure

(djenv) gregg@lindev:~/django_sandbox/fmsite$ ./manage.py makemigrations
Traceback (most recent call last):
File "./manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/home/gregg/django_sandbox/djenv/lib/python3.4/site-packages/django/core/management/init.py", line 385, in execute_from_command_line
utility.execute()
File "/home/gregg/django_sandbox/djenv/lib/python3.4/site-packages/django/core/management/init.py", line 354, in execute
django.setup()
File "/home/gregg/django_sandbox/djenv/lib/python3.4/site-packages/django/init.py", line 21, in setup
apps.populate(settings.INSTALLED_APPS)
File "/home/gregg/django_sandbox/djenv/lib/python3.4/site-packages/django/apps/registry.py", line 108, in populate
app_config.import_models(all_models)
File "/home/gregg/django_sandbox/djenv/lib/python3.4/site-packages/django/apps/config.py", line 197, in import_models
self.models_module = import_module(models_module_name)
File "/usr/lib/python3.4/importlib/init.py", line 109, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 2231, in _gcd_import
File "", line 2214, in _find_and_load
File "", line 2203, in _find_and_load_unlocked
File "", line 1200, in _load_unlocked
File "", line 1129, in _exec
File "", line 1448, in exec_module
File "", line 321, in _call_with_frames_removed
File "/home/gregg/django_sandbox/djenv/lib/python3.4/site-packages/cities_light/models.py", line 15, in
from south.modelsinspector import add_introspection_rules
ImportError: No module named 'south'

Migrations

Hallo!

Just getting prepped for deployment, and noticed that on a vanilla install, one of the migrations is broken.

http://pastebin.com/yhh6uTYU

Not sure what to do to fix it, up till now I've been lazy and whenever I run into migration problems I just delete them, which doesn't fly in production ;)

Load alternative names

The geonames data has an handy comma separeted list of alternative names for the city (items[3]), it would be fine to save it in db and configure the autocomplete integration to match there too. This would make the experience way better for cities that have a translated name in your language, eg. in italian London is Londra. I can do some of the work if you are ok with the plan.

Named indexes from geonames

Currently access to geonames item is done by index, for example item[7], but it will be useful to have named indexes, i.e. item[ICountry.code].
I've done it in commit 5d7ea32, but looks like it slows down the import process, as travis get stuck with message:

No output has been received in the last 10 minutes, this potentially indicates a stalled build or something wrong with the build itself.
The build has been terminated

I think choices are:

  • undo the commit and use simple integer indexes
  • find another way with named indexes and performance

How to only get city (not include Region, Country)

I used django-autocomplete-light va django-cities-light. But I always get City, Region, Country from CityAutocomplete. Please help me only get City (one field) ,

class RegisterForm(forms.Form):
...
city = forms.CharField(widget=autocomplete_light.TextWidget(CityAutocomplete))
state = forms.CharField(widget=autocomplete_light.TextWidget(RegionAutocomplete))
country = forms.CharField(widget=autocomplete_light.TextWidget(CountryAutocomplete))
...

When I input city, this field always city, region, country
Example:
San Jose, California, United States
Aalborg, North Denmark, Denmark
But I want to get only city: San Jose, Aalborg

Thanks and regards,
Thu Cao

New PyPI release needed

The version on PyPI (2.1.8) requires "progressbar==2.3dev", which does not exist. Therefore you cannot install this version.

This makes it a pain when it is specified, for example, as a dependency in the test suite of django-autocomplete-light

Create a signal country_items_pre_import

I think it would be good to implement a signal to import only specific countries.
In my case for example I would like to import only Brazil and United States.

I know it takes a lot of effort to get it working so I am ready to create it if you find it useful.
Just let me know if it is worth.

UnicodeDecodeError: 'ascii' codec can't decode byte 0x92

Hello,

I'm trying to run cities-light on Python 2.7 on windows.

I followed all the steps and everything works fine until I run the command "python manage.py cities_light".

i get:
Traceback (most recent call last):
File "manage.py", line 10, in
execute_from_command_line(sys.argv)
File "C:\Python27\lib\site-packages\django\core\management__init__.py", line
399, in execute_from_command_line
utility.execute()
File "C:\Python27\lib\site-packages\django\core\management__init__.py", line
392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Python27\lib\site-packages\django\core\management\base.py", line 242,
in run_from_argv
self.execute(_args, *_options.dict)
File "C:\Python27\lib\site-packages\django\core\management\base.py", line 285,
in execute
output = self.handle(_args, *_options)
File "C:\Python27\lib\site-packages\django\db\transaction.py", line 431, in in
ner
return func(_args, *_kwargs)
File "C:\Python27\lib\site-packages\cities_light\management\commands\cities_li
ght.py", line 128, in handle
geonames = Geonames(url, force=force)
File "C:\Python27\lib\site-packages\cities_light\geonames.py", line 30, in i
nit

self.downloaded = self.download(url, self.file_path, force)
File "C:\Python27\lib\site-packages\cities_light\geonames.py", line 49, in dow
nload
'%a, %d %b %Y %H:%M:%S %Z')
File "C:\Python27\lib_strptime.py", line 468, in _strptime_time
return _strptime(data_string, format)[0]
File "C:\Python27\lib_strptime.py", line 309, in _strptime
format_regex = _TimeRE_cache.compile(format)
File "C:\Python27\lib_strptime.py", line 266, in compile
return re_compile(self.pattern(format), IGNORECASE)
File "C:\Python27\lib_strptime.py", line 260, in pattern
self[format[directive_index]])
UnicodeDecodeError: 'ascii' codec can't decode byte 0x92 in position 34: ordinal
not in range(128)

I read all I could find on this matter and I understood it's probably a problem with Windows.
I deleted the file site.py (i read to do so in one of your answers) but then the error indicated that site.py was missing...
I've tried all I could think of and all I read but couldn't solve this one!

Please help!

PS: I was first using SQLite3, now i moved to MySQL with utf8_unicode_ci

Thank you for your support!

DRF HyperlinkedModelSerializer + cities-light fails

Hi,

I just installed django-citiels-light for my project using DRF and I cn't get it work.

My User Model define a city and coutry field as foreignkey, and this is what i get when i tried to get my users :
Could not resolve URL for hyperlinked relationship using view name "city-detail". You may have failed to include the related model in your API, or incorrectly configured the lookup_field attribute on this field.

Any ideas ?

Thanks !

MemoryError when running ./manage.py cities_light

Hello!

Populating cities_light table passes without problems on my development server with 8 Gb RAM. But today when I've deployed my project on production server that has 512 Mb memory, I've got an error:

$ python manage.py cities_light --force-import-all
No handlers could be found for logger "cities_light"
RAM used: 18448 kB Time: 00:00:00 Done: 100%|##################################|
RAM used: 23192 kB Time: 00:00:12 Done: 100%|##################################|
RAM used: 73952 kB Time: 00:02:40 Done: 100%|##################################|
Traceback (most recent call last):
  File "manage.py", line 10, in <module> execute_from_command_line(sys.argv)
  File "/home/django/dev.zyq/venv27/lib/python2.7/site-packages/django/core/management__init__.py", line 443, in execute_from_command_line utility.execute()
  File "/home/django/dev.zyq/venv27/lib/python2.7/site-packages/django/core/management/__init__.py", line 382, in execute self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/django/dev.zyq/venv27/lib/python2.7/site-packages/django/core/management/base.py", line 196, in run_from_argv self.execute(*args, **options.__dict__)
  File "/home/django/dev.zyq/venv27/lib/python2.7/site-packages/django/core/management/base.py", line 232, in execute output = self.handle(*args, **options)
  File "/home/django/dev.zyq/venv27/lib/python2.7/site-packages/django/db/transaction.py", line 209, in inner return func(*args, **kwargs)
  File "/home/django/dev.zyq/venv27/lib/python2.7/site-packages/cities_light/management/commands/cities_light.py", line 108, in handle geonames = Geonames(url, force=force)
  File "/home/django/dev.zyq/venv27/lib/python2.7/site-packages/cities_light/geonames.py", line 32, in __init__ self.extract(self.file_path, destination_file_name)
  File "/home/django/dev.zyq/venv27/lib/python2.7/site-packages/cities_light/geonames.py", line 71, in extract destination_file.write(zip_file.read(file_name))
  File "/usr/local/lib/python2.7/zipfile.py", line 869, in read return self.open(name, "r", pwd).read()
  File "/usr/local/lib/python2.7/zipfile.py", line 581, in read data = self.read1(n)
  File "/usr/local/lib/python2.7/zipfile.py", line 635, in read1 max(n - len_readbuffer, self.MIN_READ_SIZE)
MemoryError

How can I overcome this?

PyPI package still has dead imports

Hi, I found that Issue#63 is fixed and merged to the master branch.

However, the package at PyPI is not updated.

Could you please update the package at PyPI so future installations directly using pip install django_cities_light do not give any error?

Error in initial migrating

I've got this:

Error in migration: cities_light:0015_auto__chg_field_country_slug__chg_field_city_slug__chg_field_region_sl
DatabaseError: (1091, "Can't DROP 'cities_light_city_610a2ab0'; check that column/key exists")

Failing south migration

After I updated django to 1.6.2 and south to 0.8.4 I always get NoMigrations exception
I use django-cities-light==2.1.11. I got this same issue with a different project..
Here is a stacktrace

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/var/lib/jenkins/jobs/spv/workspace/ve/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/var/lib/jenkins/jobs/spv/workspace/ve/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/var/lib/jenkins/jobs/spv/workspace/ve/local/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/var/lib/jenkins/jobs/spv/workspace/ve/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/var/lib/jenkins/jobs/spv/workspace/ve/local/lib/python2.7/site-packages/south/management/commands/migrate.py", line 111, in handle
    ignore_ghosts = ignore_ghosts,
  File "/var/lib/jenkins/jobs/spv/workspace/ve/local/lib/python2.7/site-packages/south/migration/__init__.py", line 200, in migrate_app
    applied_all = check_migration_histories(applied_all, delete_ghosts, ignore_ghosts)
  File "/var/lib/jenkins/jobs/spv/workspace/ve/local/lib/python2.7/site-packages/south/migration/__init__.py", line 81, in check_migration_histories
    m = h.get_migration()
  File "/var/lib/jenkins/jobs/spv/workspace/ve/local/lib/python2.7/site-packages/south/models.py", line 34, in get_migration
    return self.get_migrations().migration(self.migration)
  File "/var/lib/jenkins/jobs/spv/workspace/ve/local/lib/python2.7/site-packages/south/models.py", line 31, in get_migrations
    return Migrations(self.app_name)
  File "/var/lib/jenkins/jobs/spv/workspace/ve/local/lib/python2.7/site-packages/south/migration/base.py", line 64, in __call__
    self.instances[app_label] = super(MigrationsMetaclass, self).__call__(app_label_to_app_module(app_label), **kwds)
  File "/var/lib/jenkins/jobs/spv/workspace/ve/local/lib/python2.7/site-packages/south/migration/base.py", line 90, in __init__
    self.set_application(application, force_creation, verbose_creation)
  File "/var/lib/jenkins/jobs/spv/workspace/ve/local/lib/python2.7/site-packages/south/migration/base.py", line 162, in set_application
    raise exceptions.NoMigrations(application)
south.exceptions.NoMigrations: Application '<module 'cities_light' from '/var/lib/jenkins/jobs/spv/workspace/ve/local/lib/python2.7/site-packages/cities_light/__init__.pyc'>' has no migrations.

Duplicate entry when loading data to MySQL

Hey,

This is my first post on Github so let me know if I'm not explaining myself properly.

The command "python manage.py cities_light" kept giving me this error:

No handlers could be found for logger "cities_light"

So then I tried this command: python manage.py cities_light --force-import-all

Now I get this error:

IntegrityError: (1062, "Duplicate entry '1900-Daugavpils' for key 'cities_light_region_country_id_1935599b8f52157f_uniq'")

The "1900" in '1900-Daugavpils' keeps changing to a different number each time I've tried it. This small Latvian town is giving me a huge headache. Any solutions?

Thanks!

AssertionError: slug is defined before trying to ensure uniqueness

Some of the smaller cities (for example see 6943439 in cities1000.txt) have empty items[2] and their name can not be translated to anything meaningful with set_name_ascii(). This leads to failure during initial import:

Traceback (most recent call last):40 Done:
  File "/Users/max/.virtualenvs/project/lib/python2.7/site-packages/django/core/management/base.py", line 222, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/Users/max/.virtualenvs/project/lib/python2.7/site-packages/django/core/management/base.py", line 255, in execute
    output = self.handle(*args, **options)
  File "/Users/max/.virtualenvs/project/lib/python2.7/site-packages/django/db/transaction.py", line 223, in inner
    return func(*args, **kwargs)
  File "/Users/max/demo/project/django-cities-light/cities_light/management/commands/cities_light.py", line 113, in handle
    self.from_geonames(*args, **options)
  File "/Users/max/demo/project/django-cities-light/cities_light/management/commands/cities_light.py", line 205, in from_geonames
    self.city_import(items)
  File "/Users/max/demo/project/django-cities-light/cities_light/management/commands/cities_light.py", line 400, in city_import
    city.save()
  File "/Users/max/.virtualenvs/project/lib/python2.7/site-packages/django/db/models/base.py", line 546, in save
    force_update=force_update, update_fields=update_fields)
  File "/Users/max/.virtualenvs/project/lib/python2.7/site-packages/django/db/models/base.py", line 650, in save_base
    result = manager._insert([self], fields=fields, return_id=update_pk, using=using, raw=raw)
  File "/Users/max/.virtualenvs/project/lib/python2.7/site-packages/django/db/models/manager.py", line 215, in _insert
    return insert_query(self.model, objs, fields, **kwargs)
  File "/Users/max/.virtualenvs/project/lib/python2.7/site-packages/django/db/models/query.py", line 1661, in insert_query
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/Users/max/.virtualenvs/project/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 936, in execute_sql
    for sql, params in self.as_sql():
  File "/Users/max/.virtualenvs/project/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 894, in as_sql
    for obj in self.query.objs
  File "/Users/max/.virtualenvs/project/lib/python2.7/site-packages/autoslug/fields.py", line 226, in pre_save
    assert slug, 'slug is defined before trying to ensure uniqueness'
AssertionError: slug is defined before trying to ensure uniqueness

I do not have perfect solution for this issue, but there are some measures which can help:

  • Add unidecode >= 0.04.13 to setup.py. It will be automatically picked up by autoslug (see autoslug.utils) and can produce better ascii slugs for some alphabets
  • Slightly improve set_name_ascii to ignore strings which contain only spaces:
def set_name_ascii(sender, instance=None, **kwargs):
    """
    Signal reciever that sets instance.name_ascii from instance.name.

    Ascii versions of names are often useful for autocompletes and search.
    """
    name_ascii = to_ascii(instance.name).strip()
    if name_ascii and not instance.name_ascii:
        instance.name_ascii = name_ascii
  • Add fallback for slug source:
class Base(models.Model):
    @staticmethod
    def slug_source(instance):
        """
        Return source value for slug
        """
        if instance.name_ascii:
            return instance.name_ascii
        else:
            return instance.name

    slug = autoslug.AutoSlugField(populate_from=slug_source)
  • Also there is probably missing save=True in city_import():
        if not city.name_ascii:
            # useful for cities with chinese names
            city.name_ascii = items[2]
            save = True   # this statement was missing

After all these changes I was able to successfully import cities1000.txt without any failures.

Probably there are still some chances to produce empty slug, but this can be avoided by further improving slug_source function and using geoname_id as a slug if everything other fails.

I'm also experimenting with different approaches to make slugs unique (very useful for readable URLs) but so far have nothing working yet.

django-cities-light crashes on syncdb with a fresh install of Django 1.6.2, GeoDjango, on Windows

Please be advised, I get the following traceback upon a syncdb of the "cities_light" app alongside the "geodjango" pre-requisite in INSTALLED_APPS (I'm on Windows):

(funretire) C:\Users\code\workspace\funretire\src>python manage.py migrate
Traceback (most recent call last):
File "manage.py", line 10, in
execute_from_command_line(sys.argv)
File "C:\Users\code\Envs\funretire\lib\site-packages\django\core\management__init__.py", line 399, in execute_from_command_line
utility.execute()
File "C:\Users\code\Envs\funretire\lib\site-packages\django\core\management__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Users\code\Envs\funretire\lib\site-packages\django\core\management\base.py", line 242, in run_from_argv
self.execute(args, *options.dict)
File "C:\Users\code\Envs\funretire\lib\site-packages\django\core\management\base.py", line 284, in execute
self.validate()
File "C:\Users\code\Envs\funretire\lib\site-packages\django\core\management\base.py", line 310, in validate
num_errors = get_validation_errors(s, app)
File "C:\Users\code\Envs\funretire\lib\site-packages\django\core\management\validation.py", line 34, in get_validation_errors
for (app_name, error) in get_app_errors().items():
File "C:\Users\code\Envs\funretire\lib\site-packages\django\db\models\loading.py", line 196, in get_app_errors
self.populate()
File "C:\Users\code\Envs\funretire\lib\site-packages\django\db\models\loading.py", line 78, in populate
self.load_app(app_name)
File "C:\Users\code\Envs\funretire\lib\site-packages\django\db\models\loading.py", line 99, in load_app
models = import_module('%s.models' % app_name)
File "C:\Users\code\Envs\funretire\lib\site-packages\django\utils\importlib.py", line 40, in import_module
import(name)
File "C:\Users\code\Envs\funretire\lib\site-packages\cities_light\models.py",
line 15, in
from south.modelsinspector import add_introspection_rules
File "C:\Users\code\Envs\funretire\lib\site-packages\south\modelsinspector.py", line 464, in
import south.introspection_plugins
File "C:\Users\code\Envs\funretire\lib\site-packages\south\introspection_plugins__init
.py", line 5, in
import south.introspection_plugins.geodjango
File "C:\Users\code\Envs\funretire\lib\site-packages\south\introspection_plugins\geodjango.py", line 14, in
from django.contrib.gis.db.models.fields import GeometryField
File "C:\Users\code\Envs\funretire\lib\site-packages\django\contrib\gis\db\models__init
.py", line 8, in
from django.contrib.gis.db.models.manager import GeoManager
File "C:\Users\code\Envs\funretire\lib\site-packages\django\contrib\gis\db\models\manager.py", line 2, in
from django.contrib.gis.db.models.query import GeoQuerySet
File "C:\Users\code\Envs\funretire\lib\site-packages\django\contrib\gis\db\models\query.py", line 6, in
from django.contrib.gis.db.models.fields import get_srid_info, PointField, LineStringField
File "C:\Users\code\Envs\funretire\lib\site-packages\django\contrib\gis\db\models\fields.py", line 4, in
from django.contrib.gis import forms
File "C:\Users\code\Envs\funretire\lib\site-packages\django\contrib\gis\forms__init__.py", line 2, in
from .fields import (GeometryField, GeometryCollectionField, PointField,
File "C:\Users\code\Envs\funretire\lib\site-packages\django\contrib\gis\forms\fields.py", line 11, in
from django.contrib.gis.geos import GEOSException, GEOSGeometry, fromstr
ImportError: cannot import name GEOSException

Django < 1.7 does not have migrations in django.db

Updated to latest version just now, and this broke my 1.5 installation.

[savingschampion@$] out: UnknownMigration: Migration 'cities_light:0001_initial' probably doesn't exist.
[savingschampion@$] out: Traceback (most recent call last):
[savingschampion@$] out:   File "/home/savingschampion/.virtualenv/savings_champion/local/lib/python2.7/site-packages/south/migration/base.py", line 302, in migration
[savingschampion@$] out:     migration = __import__(full_name, {}, {}, ['Migration'])
[savingschampion@$] out:   File "/home/savingschampion/.virtualenv/savings_champion/local/lib/python2.7/site-packages/cities_light/migrations/0001_initial.py", line 4, in <module>
[savingschampion@$] out:     from django.db import models, migrations
[savingschampion@$] out: ImportError: cannot import name migrations

abstract models

How about to make all models are abstract and inherit them to change their behavior or add your own? Thanks

Bus error: 10

After you resolved the another issue #31 appeared another problem.
When I click on the popup to add another City ( plus signal ) the django crash.
screen shot 2013-11-03 at 12 15 31 am
screen shot 2013-11-03 at 12 15 56 am

models.py
city = models.ForeignKey(City)

forms.py
city = forms.ModelChoiceField(City.objects.all(),
widget=autocomplete_light.ChoiceWidget('CityAutocomplete'))

registry.py
import autocomplete_light

from cities_light.models import City

autocomplete_light.register(City, search_fields=('search_names',),
autocomplete_js_attributes={'placeholder': 'city name ..'})

IntegrityError: columns country_id, name are not unique

When I run this command, it's return the error message.

$ python manage.py cities_light --force-import-all
Assuming local download is up to date for http://download.geonames.org/export/dump/countryInfo.txt
Importing countryInfo.txt
RAM used: 27025408 kB Time: 0:00:00 Done: 100%|#####################################################################################################################################|
Assuming local download is up to date for http://download.geonames.org/export/dump/admin1CodesASCII.txt
Importing admin1CodesASCII.txt
IntegrityError: columns country_id, name are not unique##################################################

Migrations problem after updating packages 3.0.1 to 3.0.2

(hotelsbank) ➜  hotelsbank  ./manage.py migrate
CommandError: Conflicting migrations detected (0003_auto_20141101_1600, 0003_auto_20141120_0342 in cities_light).
To fix them run 'python manage.py makemigrations --merge'
(hotelsbank) ➜  hotelsbank  ./manage.py makemigrations --merge
Merging cities_light
  Branch 0003_auto_20141120_0342
    - Alter field search_names on city
  Branch 0003_auto_20141101_1600
    - Alter field search_names on city

Merging will only work if the operations printed above do not conflict
with each other (working on different fields or models)
Do you want to merge these migration branches? [y/N] n

/Users/lorddaedra/.pyenv/versions/hotelsbank/lib/python3.4/site-packages/cities_light/migrations/0003_auto_20141101_1600.py:

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations
import cities_light.models


class Migration(migrations.Migration):

    dependencies = [
        ('cities_light', '0002_city'),
    ]

    operations = [
        migrations.AlterField(
            model_name='city',
            name='search_names',
            field=cities_light.models.ToSearchTextField(max_length=4000, default='', db_index=True, blank=True),
            preserve_default=True,
        ),
    ]

/Users/lorddaedra/.pyenv/versions/hotelsbank/lib/python3.4/site-packages/cities_light/migrations/0003_auto_20141120_0342.py:

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations
import cities_light.models

from cities_light.settings import INDEX_SEARCH_NAMES


class Migration(migrations.Migration):

    dependencies = [
        ('cities_light', '0002_city'),
    ]

    operations = [
        migrations.AlterField(
            model_name='city',
            name='search_names',
            field=cities_light.models.ToSearchTextField(default=b'', max_length=4000, db_index=INDEX_SEARCH_NAMES, blank=True),
            preserve_default=True,
        ),
    ]

Please provide step-by-step guide, how to deal with it?...

Warning: Incorrect string value: '\xCA\xBCs al...' for column 'name' at row 1

This is the complete stack trace of the issue, generating while running "python manage.py cities_light" command:

/usr/local/lib/python2.7/dist-packages/autoslug/utils.py:30: RuntimeWarning: Argument <type 'str'> is not an unicode object. Passing an encoded string will likely have unexpected results.
return django_slugify(unidecode(value))

RAM used: 31420 kB Time: 0:00:00 Done: 100%|###################################|
Warning: Incorrect string value: '\xCA\xBCs al...' for column 'name' at row 1

Cannot install via pip: progressbar==2.3dev fails

This is what I get when I try to pip install this:

No distributions matching the version for progressbar==2.3dev (from cities-light)

This is really weird, because to me it looks like there is a matching version on PyPi. I tried with pip 1.4 and 1.5.

List index out of range when importing data

I am using Django 1.7 and any time I try running python manage.py cities_light I get stuck with this error message.

INFO Importing alternateNames.zip
Traceback (most recent call last)::  99%|#################################### |
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "f:\python-env\django-v2\lib\site-packages\django\core\management\__init__.py", line 385, in     execute_from_command_line
utility.execute()
  File "f:\python-env\django-v2\lib\site-packages\django\core\management\__init__.py", line 377, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "f:\python-env\django-v2\lib\site-packages\django\core\management\base.py", line 288, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "f:\python-env\django-v2\lib\site-packages\django\core\management\base.py", line 338, in execute
    output = self.handle(*args, **options)
  File "f:\python-env\django-v2\lib\site-packages\django\db\transaction.py", line 454, in inner
   return func(*args, **kwargs)
  File "f:\python-env\django-v2\lib\site-packages\cities_light\management\commands\cities_light.py", line 173, in handle
    self.translation_parse(items)
  File "f:\python-env\django-v2\lib\site-packages\cities_light\management\commands\cities_light.py",     line 401, in translation_parse
    if items[2] not in TRANSLATION_LANGUAGES:
IndexError: list index out of range

Give priority to certain cities

Hello there,

Is it possible for example to give priority to the cities in particular country when searching? For example, if I start typing 'San Fran' I want to get San Francisco first or for that matter cities in the US first, and then later: 'Alcazar de San Juan', 'Cabo San Lucas', and so on :)

Thanks a lot!

Missing translations

I've noticed that you missed support for translate the field names, i will put a pull request to add this.

Issues with MySQL and Cities-Light

Here is the error I get when I run south migrations for cities-light. It dies on trying to make an index for search_names, which is a blob.

I'm using MySQL, and I've tried dropping the column suggested, but I get the same error.

Thanks!

====

FATAL ERROR - The following SQL query failed: CREATE INDEX `cities_light_city_cd532746` ON `cities_light_city` (`search_names`);
The error was: (1170, "BLOB/TEXT column 'search_names' used in key specification without a key length")
! Error found during real run of migration! Aborting.

! Since you have a database that does not support running
! schema-altering statements in transactions, we have had 
 ! to leave it in an interim state between migrations.

! You *might* be able to recover with:   - no dry run output for delete_foreign_key() due to dynamic DDL, sorry
   = ALTER TABLE `cities_light_city` DROP COLUMN `search_names` CASCADE; []

! The South developers regret this has happened, and would
! like to gently persuade you to consider a slightly
! easier-to-deal-with DBMS (one that supports DDL transactions)
! NOTE: The error which caused the migration to fail is further up.
Error in migration: cities_light:0003_auto__add_field_city_search_names
Traceback (most recent call last):
  File "manage.py", line 18, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 443, in execute_from_command_line
    utility.execute()
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/django/core/management/base.py", line 232, in execute
    output = self.handle(*args, **options)
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/south/management/commands/migrate.py", line 107, in handle
    ignore_ghosts = ignore_ghosts,
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/south/migration/__init__.py", line 219, in migrate_app
    success = migrator.migrate_many(target, workplan, database)
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/south/migration/migrators.py", line 235, in migrate_many
    result = migrator.__class__.migrate_many(migrator, target, migrations, database)
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/south/migration/migrators.py", line 310, in migrate_many
    result = self.migrate(migration, database)
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/south/migration/migrators.py", line 133, in migrate
    result = self.run(migration)
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/south/migration/migrators.py", line 107, in run
    return self.run_migration(migration)
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/south/migration/migrators.py", line 82, in run_migration
    south.db.db.execute_deferred_sql()
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/south/db/generic.py", line 309, in execute_deferred_sql
    self.execute(sql)
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/south/db/generic.py", line 273, in execute
    cursor.execute(sql, params)
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 114, in execute
    return self.cursor.execute(query, args)
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 174, in execute
    self.errorhandler(self, exc, value)
  File "/usr/local/napa/release.20120822.155041/.env/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
django.db.utils.DatabaseError: (1170, "BLOB/TEXT column 'search_names' used in key specification without a key length")

UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 559: character map to <undefined>

I have Django 1.5 and Python 3.3 and I just ran pip install. Here are the problems I hit along the way

  1. force_unicode is deprecated in Django 1.5 so I had to find and replace that with force_text
  2. unicode doesnt work in Django 1.5, so again, I had to replace that with str
  3. Then finally I hit this errror: UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 559: character map to

Help is appreciated.

Translation using the alternate names data

First of all thanks for this awesome piece of software!

Im from the netherlands and noticed that I could not find my country by searching for Nederland, but could not find it. (I have the translation code in the settings) Digging a bit further I opened up the alternate names file and checked the existence of the name.
It does exist: (http://download.geonames.org/export/dump/)

1560001 2750405 nl  Nederland   1   

the 1 at the end means its is the primary pick for the language nl.

In the code (https://github.com/yourlabs/django-cities-light/blob/master/cities_light/management/commands/cities_light.py#L397) lines with more than 4 elements are ignored. This is probably why it is ignored.

The comment states this is to avoid shortnames, colloquial, and historic. On http://download.geonames.org/export/dump/ it states that those appear from the 6th value.

Is it safe to change https://github.com/yourlabs/django-cities-light/blob/master/cities_light/management/commands/cities_light.py#L397 to > 5, or is there more to it?

Just tried it with the changed code and it seems to work...

Greets,
Roel

Error populating django-cities-light

the problem is when I give ./manage.py cities-light and other choices, it throws me the following error:

I try with ./manage.py cities-light, ./manage.py cities-light --force=FORCE, ./manage.py cities-light --force-all

cmd error:

(plataforma) C:\Users\Rafael\oveu>python manage.py cities_light --force-import-a
ll
Traceback (most recent call last):
File "manage.py", line 10, in
execute_from_command_line(sys.argv)
File "C:\Users\Rafael\plataforma\lib\site-packages\django\core\management__in
it__.py", line 399, in execute_from_command_line
utility.execute()
File "C:\Users\Rafael\plataforma\lib\site-packages\django\core\management__in
it__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Users\Rafael\plataforma\lib\site-packages\django\core\management\base
.py", line 242, in run_from_argv
self.execute(_args, *_options.dict)
File "C:\Users\Rafael\plataforma\lib\site-packages\django\core\management\base
.py", line 285, in execute
output = self.handle(_args, *_options)
File "C:\Users\Rafael\plataforma\lib\site-packages\django\db\transaction.py",
line 399, in inner
return func(_args, *_kwargs)
File "C:\Users\Rafael\plataforma\lib\site-packages\cities_light\management\com
mands\cities_light.py", line 128, in handle
geonames = Geonames(url, force=force)
File "C:\Users\Rafael\plataforma\lib\site-packages\cities_light\geonames.py",
line 30, in init
self.downloaded = self.download(url, self.file_path, force)
File "C:\Users\Rafael\plataforma\lib\site-packages\cities_light\geonames.py",
line 49, in download
'%a, %d %b %Y %H:%M:%S %Z')
File "c:\Python27\Lib_strptime.py", line 467, in _strptime_time
return _strptime(data_string, format)[0]
File "c:\Python27\Lib_strptime.py", line 308, in _strptime
format_regex = _TimeRE_cache.compile(format)
File "c:\Python27\Lib_strptime.py", line 265, in compile
return re_compile(self.pattern(format), IGNORECASE)
File "c:\Python27\Lib_strptime.py", line 259, in pattern
self[format[directive_index]])
UnicodeDecodeError: 'ascii' codec can't decode byte 0xed in position 23: ordinal
not in range(128)

Tables are created with south normally.

I'm using python 2.7 and django 1.6.2, south.

that I can do to fix it?

Thank

Call for contributor: Implement optionnal support for GeoDjango

There has been a couple of forks to add GeoDjango support. So it would make sense to add GeoDjango support, as long as it stays optionnal.

The way I see it:

  • in settings.py, add a configuration variable that would check the database configuration dict for geodjango support
  • in models.py, add PointFields if that variable is True

So if you're going to implement this for your project, why not consider this pattern ?

Python 3.3 and django-cities-light on Windows 8.1 don't work.

When I run "manage.py cities_light" django-cities-light dies.
C:\Git\XXX\src>manage.py cities_light
RAM used: ?? kB Time: 0:00:00 Done: 100%|#####################################|
Traceback (most recent call last):
File "C:\Git\XXX\src\manage.py", line 10, in
execute_from_command_line(sys.argv)
File "C:\Python33\lib\site-packages\django\core\management__init__.py", line
385, in execute_from_command_line
utility.execute()
File "C:\Python33\lib\site-packages\django\core\management__init__.py", line
377, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "C:\Python33\lib\site-packages\django\core\management\base.py", line 288,
in run_from_argv
self.execute(_args, *_options.dict)
File "C:\Python33\lib\site-packages\django\core\management\base.py", line 338,
in execute
output = self.handle(_args, *_options)
File "C:\Python33\lib\site-packages\django_cities_light-3.0.3-py3.3.egg\cities
_light\management\commands\cities_light.py", line 156, in handle
progress = progressbar.ProgressBar(maxval=geonames.num_lines(),
File "C:\Python33\lib\site-packages\django_cities_light-3.0.3-py3.3.egg\cities
_light\geonames.py", line 98, in num_lines
return sum(1 for line in open(self.file_path))
File "C:\Python33\lib\site-packages\django_cities_light-3.0.3-py3.3.egg\cities_light\geonames.py", line 98, in
return sum(1 for line in open(self.file_path))
File "C:\Python33\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 404: character maps to

Python 3.3.5 (v3.3.5:62cf4e77f785, Mar 9 2014, 10:37:12) [MSC v.1600 32 bit (In
tel)] on win32
And latest django from pip on 2015-01-25.

Any ide on how to solve this?

0003_auto__add_field_city_search_names migration fails in mysql

The migrations fails in mysql 5.0 with:

_mysql_exceptions.OperationalError: (1170, "BLOB/TEXT column 'search_names' used in key specification without a key length")

removing db_index=True from the model field fix the issue for me. How should this handled?

Is there an easy way to get custom fields in the Country model?

Great job on this app. It fills a big hole in easily getting geonames data into Django sites with postgis. The site I'm working on now has a requirement to have Countries ODA Recipients status and UN Subregion listed. Is there an easy way to add custom fields the Country model? Both of these values aren't in geonames so I can add that information myself after the geonames information has been imported.

I briefly tried to get something going by sub-classing the Country model and adding the fields I needed. That didn't work too well because the Country model was created and not the sub-classed model when using the cities_light management command. I could use a signal to create an additional model to hold the info I need when the Country model created (like User and UserProfile) but I'd still have an awkward admin interface with data in two models. Is there a better way I can accomplish this task or would it better to just fork the repo and add the extra fields I want?

Thanks, Ben

  1. http://www.oecd.org/dac/aidstatistics/daclistofodarecipients.htm
  2. https://en.wikipedia.org/wiki/Subregion#United_Nations_subregions

DRF 3.0 compatibility

Hi,

It would be great to have DRF 3.0 compatibility.

Apparently from the first test i did, you just have to add the queryset attribute on custom Fields. (except url)

Thanks !

django-cities-light support django 1.7 ?

I used django version 1.7/MySQL 5.5.38 InnoDB on ubuntu, my settings:
INSTALLED_APPS = (
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',

'autocomplete_light',
'testdjango',

)

if django.VERSION < (1, 7):
INSTALLED_APPS += ('south',)

SOUTH_MIGRATION_MODULES = {
'cities_light': 'cities_light.south_migrations',
}

python manage.py syncdb

DEBUG:django.db.backends.schema:CREATE INDEX cities_light_city_d7397f31 ON cities_light_city (name_ascii); (params [])
DEBUG:django.db.backends:(0.030) CREATE INDEX cities_light_city_d7397f31 ON cities_light_city (name_ascii); args=[]
DEBUG:django.db.backends.schema:CREATE INDEX cities_light_city_2dbcba41 ON cities_light_city (slug); (params [])
DEBUG:django.db.backends:(0.028) CREATE INDEX cities_light_city_2dbcba41 ON cities_light_city (slug); args=[]
DEBUG:django.db.backends.schema:CREATE INDEX cities_light_city_b068931c ON cities_light_city (name); (params [])
DEBUG:django.db.backends:(0.030) CREATE INDEX cities_light_city_b068931c ON cities_light_city (name); args=[]
DEBUG:django.db.backends.schema:CREATE INDEX cities_light_city_610a2ab0 ON cities_light_city (search_names); (params [])
DEBUG:django.db.backends:(0.000) CREATE INDEX cities_light_city_610a2ab0 ON cities_light_city (search_names); args=[]
Traceback (most recent call last):
File "manage.py", line 10, in
execute_from_command_line(sys.argv)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/core/management/init.py", line 385, in execute_from_command_line
utility.execute()
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/core/management/init.py", line 377, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/core/management/base.py", line 288, in run_from_argv
self.execute(_args, *_options.dict)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute
output = self.handle(_args, _options)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/core/management/base.py", line 533, in handle
return self.handle_noargs(
_options)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/core/management/commands/syncdb.py", line 27, in handle_noargs
call_command("migrate", *_options)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/core/management/init.py", line 115, in call_command
return klass.execute(_args, *_defaults)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/core/management/base.py", line 338, in execute
output = self.handle(_args, *_options)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 160, in handle
executor.migrate(targets, plan, fake=options.get("fake", False))
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 63, in migrate
self.apply_migration(migration, fake=fake)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/db/migrations/executor.py", line 97, in apply_migration
migration.apply(project_state, schema_editor)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/db/backends/schema.py", line 82, in exit
self.execute(sql)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/db/backends/schema.py", line 98, in execute
cursor.execute(sql, params)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 81, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/db/utils.py", line 94, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 128, in execute
return self.cursor.execute(query, args)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/home/ubuntu/.virtualenvs/testdjango/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
django.db.utils.OperationalError: (1170, "BLOB/TEXT column 'search_names' used in key specification without a key length")

Populating DB time

Hey there,

Trying to work with the library.
Running python 2.7.3 and Django 1.5.5 with Postgresql.
I'm using South as well.
I've migrated using south and created the tables.
Now when I run the "cities_light" command its does start to populate but after running more then an hour the ETA is 8 more hours.

This doesn't seem like a legit time.. Am I doing something wrong? is there a way to speed things up?

Best

Problems with encoding using MySQL

Hey @jpic

First, thanks for the work! :-)

I remember the last time I used django-cities-light everything was up and running here.

I use Centos 6.4 on production machines and it uses Python 2.6. Is Python 2.6 still supported?

I am having a hard time trying to load django-cities-light here. Follows the traceback:

/usr/local/venv/myenv/lib/python2.6/site-packages/zerorpc/events.py:31: DeprecationWarning: gevent.coros has been renamed to gevent.lock
  import gevent.coros
/usr/local/venv/myenv/lib/python2.6/site-packages/autoslug/utils.py:30: RuntimeWarning: Argument <type 'str'> is not an unicode object. Passing an encoded string will likely have unexpected results.
  return django_slugify(unidecode(value))
Traceback (most recent call last):
  File "manage.py", line 19, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/cities_light/management/commands/cities_light.py", line 169, in handle
    self.country_import(items)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/cities_light/management/commands/cities_light.py", line 256, in country_import
    self.save(country)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/cities_light/management/commands/cities_light.py", line 495, in save
    model.save()
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/django/db/models/base.py", line 545, in save
    force_update=force_update, update_fields=update_fields)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/django/db/models/base.py", line 573, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/django/db/models/base.py", line 654, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/django/db/models/base.py", line 687, in _do_insert
    using=using, raw=raw)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/django/db/models/manager.py", line 232, in _insert
    return insert_query(self.model, objs, fields, **kwargs)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/django/db/models/query.py", line 1514, in insert_query
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/django/db/models/sql/compiler.py", line 903, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/mysql/connector/django/base.py", line 133, in execute
    return self._execute_wrapper(self.cursor.execute, query, args)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/mysql/connector/django/base.py", line 116, in _execute_wrapper
    utils.ProgrammingError(err.msg), sys.exc_info()[2])
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/mysql/connector/django/base.py", line 113, in _execute_wrapper
    return method(query, args)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/mysql/connector/cursor.py", line 480, in execute
    stmt = operation % self._process_params(params)
  File "/usr/local/venv/myenv/lib/python2.6/site-packages/mysql/connector/cursor.py", line 355, in _process_params
    "Failed processing format-parameters; %s" % err)
django.db.utils.ProgrammingError: Failed processing format-parameters; Python 'safetext' cannot be converted to a MySQL type

I connected to my database and I checked the charset using the following command: SELECT DEFAULT_CHARACTER_SET_NAME, DEFAULT_COLLATION_NAME FROM INFORMATION_SCHEMA.SCHEMATA WHERE SCHEMA_NAME = 'azionmanager02_hlg';

Result was:

DEFAULT_CHARACTER_SET_NAME : utf8
DEFAULT_COLLATION_NAME : utf8_general_ci

Am I missing something?

Should the slug for City, Region and Country be unique?

I am not sure if i am confused, but i understand that the base model define the field "slug" with the intention of being unique. But for example for cities at least it is not, a search for "valencia" as the slug returns 4 results.

When doing a detail view searching a city by the slug field, returns an error, because it finds 4 objects and not one.

django-cities-light

So, shouldnt the "slug" field be define as unique?

Thanks very much by the way, think i will be using this, so i hope to help around

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.