hasadna / open-knesset Goto Github PK

need to switch to Use the Truncator class in django.utils.text instead.

we need an easy way to export MK, vote and bill data to excel (csv).

API v2 supports that. see the API docs: https://oknesset-api.readthedocs.org/en/latest/index.html . add ?format=csv (not documented yet).

Just need to create a link from MK list page, vote list page, and bill list page to the respective API URL (note that we would like to preserve the user's filters if possible. so for example, the page https://oknesset.org/bill/?stage=2 should link to oknesset.org/api/v2/bill/?stage=2&format=csv&limit=0 (the limit=0 is to disable pagination - give all results in 1 file).

in pages like http://oknesset.org/tags/%D7%AA%D7%97%D7%91%D7%95%D7%A8%D7%94%20%D7%A6%D7%99%D7%91%D7%95%D7%A8%D7%99%D7%AA/ there is a list of related committee meetings. we need to:

1. add the name of the committee (above the topics)
2. add the number of participating mks (below the date)

in all lists (committee meetings, votes, bills) we need to present only 20 items by default, with a button to see more.

for example http://oknesset.org/committee/3/all_meetings/ is supposed to present only meetings of committee 3 (and it says so in the title) but it presents meetings by all committees.

for example, in http://oknesset.org/committee/meeting/7246/ and http://oknesset.org/committee/meeting/7294/ MK Elazar Shtern is found by the algorithm, when in fact it is the committee legal advisor Elazar Shtern.

This confused us earlier when we ran a search in a dev server and suddenly found ourselves on the production server.

This is caused by the results coming as is from the google custom search. I don't know if it's worth the hack, but perhaps we should rewrite the urls to be relative before displaying them.

The downside to this is that there's a good chance that the linked page isn't part of the dev db, and will be broken, so feel free to mark as "won't fix".

For example, in here: http://oknesset.org/committee/4/

in committee meeting pages (e.g. https://oknesset.org/committee/meeting/7313/ ), there are links with text האירו. when a user with no edit permissions (no validated email) clicks it, nothing happens.

It should display a message saying you need to validate your email with a link to the user profile.

1. I need the stage of the vote, "approved", "first call" etc... (see http://oknesset.org/api/v2/vote/3335)
2. In the agenda response some filds are part of the vote entity but most of them are missing (like http://oknesset.org/api/v2/agenda/41/)

Thanks,
Alon

See in the following image:

• The menu should be centred, preferably without the right border.
• The box shadow is visible on the sides, remove it for small screens.
• The location of the search icon is outside of the search box.
• There's lots of spacing between the seach box and next header, but no spacing between twitter and facebook icons and "פרוייקט של הסדנא לידע ציבורי".
• This should happen only on small screens. The design should stay as is for large (desktop) screens.

‎Tips

• Make sure to check on smartphones as well.
• Firefox has "Responsive Design View" accessible by the shortcut Ctrl+Shift+M.
• Don't edit app.css directly, edit the less files and compile the final css. See: Working on CSS in our developer docs.

in pages like http://oknesset.org/committee/3/all_meetings/ , we need to add more filtering options:

1. show only meetings with up to X attending MKs.
2. show only meetings with X or more attending MKs.
3. show only meetings that have no tags.
4. show only meetings with הצעת חוק in the title, but no lined bill.
5. show meeting where specific MKs attended

pages like http://oknesset.org/party/by/bills-approved/ and http://oknesset.org/member/by/bills_proposed/ show data that is based on all bills. this needs to be changed to count only bills that were proposed/approved/etc in the current knesset.

The visualization is supposed to present the supported in green from the top clockwise (to the right) and the opposed in red from the top counter-clockwise (to the left). It currently doesn't work like this.

should be:

This should be addressed or ideally fixed by @ydaniv

https://github.com/hasadna/Open-Knesset/

Suggestion:

Activate the virutalenv cd oknesset; . bin/activate Note the changed prompt which includes the virtualenv's name.

""" add:
Fork the repository [or do whatever else needs to be done for the rest to work]
""" and then continue with:
Clone the repository:

git clone https://github.com/your-username/Open-Knesset.git
This creates a copy of the project on your local machine.

Current tagging situation is a mess and should be improved.

• Tags Relationships:
  • FK to the master tag
  • When someone tags with a "child" tag, show notification that it was tagged as the master tag
  • Auto complete on all tags (Master and children)
  • django-taggit with our custom tag model
  • django-tagging remains for the planet functionality
• New tags:
  • Users can't create tags no more
  • Should go through the suggestions mechanism, and approved by the editors
• API
  • Expose the taggit models

Vote API should use the same filters format as out /vote/ page, because in /vote/ page we have an "export to csv" button. For example, after setting the filter to a date range, the "export to csv" link seems to update correctly to http://oknesset.org/api/v2/vote/?from_date=2012-03-01&tagged=all&vtype=all&order=time&to_date=2012-03-15&format=csv&limit=0 but the resulting file still contains votes outside the specified range. The same happens if you change the format to json.

related issue: there is a limit=0 param, but it is also ignored, and 1000 results are returned.

In bills the issue is similar, but stage=all seems to cause an empty result list (and the rest of the filters are ignored).

This type of page http://oknesset.org/bill/?member=826 exists, but the only way to get to it is from page like http://oknesset.org/member/by/bills_proposed/ (see also the other stage filters) Need to add it to the bill list filter, and then link from each MK page to the list, filtered to show bills proposed by that MK (possibly, add links to the list filtered by different bill stages - proposed, passed pre vote, etc.).

in https://oknesset.org/member/739/%D7%A2%D7%9E%D7%A8%D7%9D-%D7%9E%D7%A6%D7%A0%D7%A2/ the role description says:
יו"ר ועדת החינוך | התרבות והספורט
as if יו"ר ועדת החינוך is one role, and התרבות והספורט is another role.

the is caused by the roles property (mks/models.py:359) splitting the role string by comma. this specific role happens to contain a comma.

suggested fix: change to splitting by some other character (e.g, pipe - |), and make sure the scraping code concatenates different roles with | in between. use content people to help if needed.

Search the web for academic paper which are writing in Open Knesset, in order to make a bank of references for our use.

go to login page, enter incorrect username / password. you'll get the error message:
"Please enter a correct שם משתמש and password. Note that both fields may be case-sensitive"
which is partially translated.
need to add django he translations to our po/mo files.

.במשימה זו יש לדבר עם עפרי, מוביל הפרוייקט או לעשות ביחד עם מתכנת, ולהסביר מה אומרת כל סטטיסיטיקה על פעילות ח"כים בכנסת פתוחה
:בדף ח"כ

נוכחות במשכן
נוכחות בועדות
משמעת סיעתית
משמעת קואליציה

Open Knesset's About Page contains a box of mentions in the media.

The content is not up to date, with the latest entry dated March, 2011. Please provide more links to be included in that box.

For example here: http://oknesset.org/committee/topic/32/
in the "הפיצו" box.

all the suggested feedback objects have /feedback/ as their URL.
The URL should contain the URL which the user was at, when clicking the feedback button (to help the person reading the feedback get the proper context).

in pages like https://oknesset.org/vote/5894/ if the logged in user is an editor of an agenda, he gets a form to link the current vote to his agenda(s). The form is submitted to a generic view (update_editors_agendas in agendas/view.py) that deals with several types of similar forms. After a form is submitted, we return a redirect. if the form is invalid (for example, the 'reasoning' field contains more than the allowed 1000 chars), we don't show any error message.

This view should render the appropriate view (vote detail, bill detail, or committee meeting detail, according to the one that sent the form), with a bound form, to display the errors to the user.

Open Knesset has this page on Hasadna wiki - http://wiki.hasadna.org.il/index.php?title=%D7%9B%D7%A0%D7%A1%D7%AA_%D7%A4%D7%AA%D7%95%D7%97%D7%94

Need to add:

• Contact details
• Technical details
• What volunteers need to do in the project
• History of the project
• Links to media coverage
• etc.

Make sure agenda actually has an image in laws/api.py#91.

Example URL from tracebacks:
/api/v2/vote/681/

for example, in http://oknesset.org/member/105/%D7%A2%D7%9E%D7%99%D7%A8-%D7%A4%D7%A8%D7%A5/ , click the question mark near משמעת קואליציה and you get a list of votes this MK voted against the coalition. its computed wrong - because in the previous knesset he was in a different party than other MKs in his current party, it is counted as if he voted against them, but they were not really his party/coalition at the time of those votes.

There is no native search option within the Open Knesset database. The current google search is not good enough. Information is not accessible because of this.

currently, in pages like http://oknesset.org/committee/5/ , in the mmm documents section we show all documents, and not sorted by anything obvious. need to sort by date (recent first) and show only documents from last 6 months, with an option to click and get more.

Your mission, should you choose to accept it, is to write a clear explanation for the סיעות metric found in the אג'נדות section of the website, so that the layman oknesset user will be able to make sense of what this metric means.

(link to image:
https://googledrive.com/host/0B09gtBbVkTqnN1lQc2ZubkkyTm8/%D7%90%D7%92'%D7%A0%D7%93%D7%95%D7%AA_%D7%A1%D7%99%D7%A2%D7%95%D7%AA.png)

Please include:

a. A simple clarification of what this metric measures.

b. Where the data collected to create this metric comes from.

c. How we analyze/manipulate the data to arrive at the metric.

In case you've chosen to tackle this mission, please indicate you've done so in the comments section.

Once your done, please upload the explanation in the comments section, and indicate where you received the information from, and when.

Thanks!

example: in /api/v2/vote/685/

File "/oknesset_web/oknesset/Open-Knesset/laws/api.py", line 91, in dehydrate_agendas
    'image': agenda.image.url,

  File "/oknesset_web/oknesset/local/lib/python2.7/site-packages/django/db/models/fields/files.py", line 63, in _get_url
    self._require_file()

  File "/oknesset_web/oknesset/local/lib/python2.7/site-packages/django/db/models/fields/files.py", line 41, in _require_file
    raise ValueError("The '%s' attribute has no file associated with it." % self.field.name)

ValueError: The 'image' attribute has no file associated with it.

need to deal with agenda.image is None case.

the code is in simple.management.commands.syncdata:936

it works OK for most committee meeting protocols, but for some, it returns an empty string.

to recreate the problem run in a django shell:

from simple.management.commands.syncdata import Command
x = Command()
y = x.get_committee_protocol_text('www.knesset.gov.il/protocols/data/rtf/huka/2013-10-08.rtf')
y
Out: ''

You should get a very long unicode of the protocol text, and not an empty string. This specific RTF opens OK in LibreOffice, so its not too badly broken.

as an example of an RTF that is scrapped correctly, you can use
y = x.get_committee_protocol_text('http://www.knesset.gov.il/protocols/data/rtf/kalkala/2013-10-09.rtf')

leads to a blank page.

Card titles are built to present an icon next to their title. This has been a part of Adi's original design. We need to conclude the icon design and to crop them to the right sizes.

The link to distribute in the Windows section of the developers documentation is no longer valid.

Alternate means of installing distribute is required, and documentation update to match it.

Right now the homepage slider only shows tidbits, which creates sad incidents of ugly huge banners in the homepage. We need to:

• Allow marketing messages (aka banners) in the slider
• Finish the design of the slider
• limit the number of characters in an item
• define some rules about the dynamic image (for example, use MK profile image on items concerning them)

After defining and designing these items, they can be turned into new issues to be implemented separately.

in vote, such as http://oknesset.org/vote/6196/ it would be nice to have a link to the relevant part in the plenum protocol http://oknesset.org/plenum/7678/ - the correct plenum meeting can be easily found by date (there is only 1 plenum protocol per day. possible edge case - votes that are after midnight might be in the plenum protocol of the previous date! need to check this). the relevant part inside the protocol may be found by using vote.vote_number, and matching it against a protocol part header (see below).

from datetime import date
p = Committee.objects.get(name='Plenum')
// p is the plenum "committee" object.
m = p.meetings.get(date=date(2013, 03, 19))
// m is the meeting object. m.protocol_text is a bulb of text of the protocol, m.parts is the protocol, parsed to small chunks.

parts = m.parts.filter(header__contains=u'הצבעה')
// these are all the parts that have הצבעה in their name. each should also have a number. just find the one where the number matches vote.vote_number. we'll assume we put it in the variable part
part.get_absolute_url should give you the direct link to it.

The link should be added using the Links app. need to make sure it is presented in the vote page.
Link.objects.create(...)

This whole thing should be written as a management command, with options to run it on all the votes in the db, or just on votes from last x days.

when reviewing user feedback, we need the editors to be able to send a short message, preferably by email (or by in-site notification if we don't have the email - this is optional) to say thank you, to explain why something won't be implemented, ask further questions, etc.

The map here (gist) uses Oknesset API for place_of_residence* and additional MK information.

We've discovered various errors there (based on a small sample of MKs and ex-MKs):

• In some cases (e.g. all MKs with residence in Jerusalem), several MKs get the same geodata (we had to add noise to make the map more usable).
• For some MKs, the textual place_of_residence doesn't match the geodata. For example, Yariv Levin:
  
• Some MKs are "homeless" 😉.

Does the place_of_residence* information come straight from a gov site?

If it does — it could help if we could quote a gov URL to people who ask where the data comes from.
If not (i.e. it's maintained at some hasadna project) — is there a procedure to suggest corrections?

Kavod,
The Dod

oknesset.org/api/v2/vote/?format=csv&limit=50 exports votes data. it does not provide column headers, and it should.

code in simple.managements.sync_data : get_approved_bill_text

to recreate the problem, use
from simple.management.commands.syncdata import Command
x = Command()
x.get_approved_bill_text('http://www.knesset.gov.il/privatelaw/data/19/3/711_3_1.rtf')

UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 0: character maps to

the knesset apis have information about lobbyists - see how we can integrate it into open knesset

http://online.knesset.gov.il/WsinternetSps/KnessetDataService/LobbyistData.svc/

I was reading today articles in the news about babies left in the car. I then stumbled upon this link: http://oknesset.org/committee/topic/32/

I couldn't tell when this topic was written nor when the comments on it was submitted.

The plenum protocols are very long and hard to read.

The protocol can be divided into sections by detecting headers in the text.

Can add a UI element for navigating between those sections.

When the text is too long, the button seems to run over and hide a part of the text.

יש לתקן את דברי ההסבר למשתמש בדף ההצבעה לדוגמא כאן:
: http://oknesset.org/vote/5678/
(הטקסט שמסומן בכחול)
יש לשים לב כי הדף עוסק על הצבעות במליאה. הצבעות במליאה לא חייבות להיות על הצעות חוק, אלא למשל על מינויים שונים בכנסת, אי אמון וכו'.