sammitjain / loksabha-questions Goto Github PK

There are ~ 7 questions that show up with null values in contents which seems to be an error (The core assumption being that a question asked in the Lok Sabha will have some body)

This is likely due to the question elements being excluded by the PDF Parser.

For the last starred/unstarred question of a session, the PDF parser picks up contents till the end of page. This causes it to include Secretary name, Date and Place as well. This information can be removed from the question contents as part of the clean-up process.
(Utpal Kumar Singh, Date, New Delhi)

Fuzzy name matching of MP names is prone to error even with a high match score.
Consider the following ambiguous names which are all valid MP names but were mapped to a single name - Kumar, Shri Sunil:

• Shri Sunil Kumar Singh
• Shri Sunil Kumar
• Shri Sunil Kumar Mondal
• Shri Sunil Kumar Soni
• Shri Sunil Kumar Pinto

Consider preprocessing topic contents in the questions dataset