mysociety / yournextrepresentative Goto Github PK
View Code? Open in Web Editor NEWA website for crowd-sourcing structured election candidate data
Home Page: https://candidates.democracyclub.org.uk/
License: GNU Affero General Public License v3.0
A website for crowd-sourcing structured election candidate data
Home Page: https://candidates.democracyclub.org.uk/
License: GNU Affero General Public License v3.0
Add a scraper or scrapers for Green Party candidates, similar to these ones:
The maximum person ID in use should now be set by the candidates_create_popit
command, but currently creating new people doesn't use this information. It needs to try the ID MaxPopItIds.get_max_persons_id() + 1
first, increment it until creation succeeds, and then use MaxPopItIds.update_max_persons_id(...)
to record the ID that worked.
Should users be able to alter how their name appears on the site? Is it possible for 2 people to register with the same user name? And does it matter?
(And do I need [an admin] to change mine?)
So you can change a Wikipedia URL or Twitter username for a person, or add one where there was no previous value, but clearing the field and submitting it has no effect. Presumably we're not sending that field to PopIt in that case, and then PopIt's unexpected PUT behaviour (patching, not replacing) hits us...
At the moment there's no way to see information about a candidate except by clicking "Edit" beside them (and that's only available to logged-in users). We should add even a very basic page that displays the information we have for that candidate.
Add a scraper or scrapers for UKIP candidates, similar to these ones:
In the 2010 election, Graham Lee http://yournextmp.mysociety.org/person/804 unsuccessfully stood as a candidate for Ruislip, Northwood and Pinner.
The Green Party PPC page for London (http://london.greenparty.org.uk/elections/2015-general-election.html - the last (incomplete) entry) currently has him listed for neighbouring Uxbridge and South Ruislip instead.
Assuming that the Green Party page is correct, how should we handle that? Maybe a search for candidates of the same name when creating a new candidate might be part of it (there are currently 2 "Graham Lee"s in the live data as I'd forgotten he stood elsewhere last time around - sorry - http://yournextmp.mysociety.org/person/4547 is the duplicate)
On http://mark.candidates.dev.mysociety.org/person/stephen-williams/update I added a homepage url and twitter username (which appear to have saved correctly) but got the following error.
Environment:
Request Method: POST
Request URL: http://mark.candidates.dev.mysociety.org/person/stephen-williams/update
Django Version: 1.6.5
Python Version: 2.7.3
Installed Applications:
('django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django_nose',
'pipeline',
'candidates',
'debug_toolbar')
Installed Middleware:
('django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware')
Traceback:
File "/data/vhost/mark.candidates.dev.mysociety.org/venv/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
112. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/data/vhost/mark.candidates.dev.mysociety.org/venv/lib/python2.7/site-packages/django/views/generic/base.py" in view
69. return self.dispatch(request, *args, **kwargs)
File "/data/vhost/mark.candidates.dev.mysociety.org/venv/lib/python2.7/site-packages/django/views/generic/base.py" in dispatch
87. return handler(request, *args, **kwargs)
File "/data/vhost/mark.candidates.dev.mysociety.org/venv/lib/python2.7/site-packages/django/views/generic/edit.py" in post
171. return self.form_valid(form)
File "/data/vhost/mark.candidates.dev.mysociety.org/yournextmp-popit/candidates/views.py" in form_valid
540. person.id
File "/data/vhost/mark.candidates.dev.mysociety.org/yournextmp-popit/candidates/views.py" in set_party_membership
254. person = PopItPerson.create_from_popit(self.api, person_id)
File "/data/vhost/mark.candidates.dev.mysociety.org/yournextmp-popit/candidates/models.py" in create_from_popit
79. new_person._update_organizations()
File "/data/vhost/mark.candidates.dev.mysociety.org/yournextmp-popit/candidates/models.py" in _update_organizations
91. for m in self.popit_data['memberships']:
Exception Type: KeyError at /person/stephen-williams/update
Exception Value: 'memberships'
Lots of these are arty shots with lots of landscape, so we'd want to use OpenCV to crop to faces, as SayIt does now.
This should probably be a nice error message instead.
It's confusing having this separate at the moment.
Tnks
When inputting a date of birth, I got a 'please enter a valid date' message, but there's no indication of the approved format.
(I'd pasted straight from Wikipedia as "December 22, 1964").
I need to create new Facebook, Google and Twitter keys / secrets for social logins for the live site.
We now have a list of registered parties from the Electoral Commission in PopIt so it seems sensible just to offer those as a choice, rather than the current behaviour when anything entered that isn't the same as something in the autocomplete will cause a new party to be created. (This probably would result in lots of typo or alias parties being created.)
These party logos are in PopIt now (e.g. see https://yournextmp.popit.mysociety.org/organizations/party:52) so we could display them alongside the candidates. (The image proxy should let you request them in a standard thumbnail size, as well, I think.)
Suggested by @symroe
LinkedIn pages - e.g. https://www.linkedin.com/pub/layla-moran/28/670/284 are an incredibly rich source of background information on candidates. A field should be added to enable these to be populated for the candidate records
Having created a new candidate with an error in the gender field (copy and paste/not paying attention problem), it's now not available to correct in the edit screen.
The "versions" data in PopIt helps you to see what happened to a particular candidate's record, but it doesn't provide an easy view of what particular users have done (and wouldn't record deletion of people, for example). If we get a malicious user, it would also be helpful to be able to go through all the changes they've made to revert them as well. So, a simple "action log" table would help with this.
There are corrections that people might want to make to the site that they either can't figure out how to do, or aren't currently possible in the interface - the main one of these that strikes me is indicating that two different people should be merged (#25). If there was a feedback form on each page (as with Pombola) with a captcha, then we could collect these corrections to apply.
Currently the admin interface drop down selection lists all parties alphabetically. There are hundreds.
Instead it is proposed that the top five parties are listed at the top alphabetically, with the remainder beneath a dotted line:
Conservatives
Green
Labour
Lib Dem
UKIP
In other words, only the numeric ID matters, but it makes the URLs more easily recognisable.
This might be useful for simple demographic analysis
At the same time, make sure that if a person pastes in a Twitter URL when entering data, we just extract the username.
... after talking to Edmund about changing the DNS.
From @symroe
after updating her she has gone in the "known
not to be standing again" category. This is inferred by me making an edit
and not checking the 'standing in 2015' box, but I never actually stated
that she will not be standing again (I'm sure she will be, but I can't find
a source for that). Maybe we should only mark people as not standing if
there is an explicit statement to that effect?
Because of the way I imported the data, the image URLs all refer to localhost URLs - I've added a candidates_delete_party_images
command and then we can just rerun candidates_update_popit_parties_from_ec_data
.
At the moment I'm thinking that this is potentially so open to abuse (and was problematic for YourNextMP in 2010) that we'd want to restrict the users who can do it, or have a submitted photos moderation queue.
You can get Javascript autocompletion of party names directly from PopIt (using JSONP) - the logic for forming the URL would look a bit like this:
But it might be helpful to provide a JSONP-accessible API endpoint for this in the front-end as well. This could be helpful for people as well as parties.
A confirmed email address is required even if you're using social login, which is probably excessive. The reason for this at the moment is just not having found a satisfactory combination of django-allauth settings, and wanting to err on the side of discouraging any malicious users at first...
Comments from my earlier email
- Login with twitter is needlessly complex: we don't need to verify email,
I don't think. (I've not tried the other methods)Yeah, it's way over the top, but I couldn't find any other
combination of django-allauth settings that were satisfactory,
and at the moment I'm more worried about junk data being added
by time-wasters than not having enough people take part. [...]
There's some comments about that change to the authentication
here:As I said there, I'm tempted to ditch the social login entirely
(even though I like the idea of storing less user data), partly
to make this simpler and partly because I keep reading things
like this:https://medium.com/on-coding/the-unexpected-costs-of-third-party-login-cda41c087653
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.