pombreda / googletransitdatafeed Goto Github PK

In IE6 the bottom panel doesn't display at all. In FF3 it appears to not
resize after drawing the marey graph, leaving the graph below the window
bottom until the browser window is resized.

Currently, Schedule Viewer doesn't display shapes for trips if they're
present, making it more difficult to evaluate/debug shape data

This is what the trace looks like:

Traceback (most recent call last):
  File "./feedvalidator.py", line 64, in ?
    loader.Load()
  File "/home/jhughes/gtdf/trunk/python/transitfeed.py", line 2271, in Load
    self._LoadCalendar()
  File "/home/jhughes/gtdf/trunk/python/transitfeed.py", line 2049, in
_LoadCalendar
    period = ServicePeriod(field_list=row)
  File "/home/jhughes/gtdf/trunk/python/transitfeed.py", line 1203, in __init__
    self.original_day_values += [value.strip()]
AttributeError: 'NoneType' object has no attribute 'strip'

It would be nice to have the feed validator add the following things to its 
output:
* Agency Name
* Stop/Route/Trip count
* Effective date range

(On behalf of TomH)

In Trapeze, each actual stop pattern comes from (is a subset of) the master
pattern (the entirely of the master pattern may never be used in any actual
stop pattern). The sequence numbers come from the master pattern.

We should only detect a bad sequencing if a stop is missing for all trips
of a route.

An example,
A master pattern and sequencing.
stop_id     sequence
a              1
b              2
c              3
d              4

Applying the master pattern to a trip which only services b-c-d,
stop_id     sequence
b              2
c              3
d              4
will generate a sequence error.

We need to add validation to ColorLuminance or things that call it.

It would be great to have a web-based version of feedvalidator that would
make it ridiculously easy to check the integrity of a feed.

Do something cool with http://graphserver.sourceforge.net/
It might be possible to rewrite the Schedule class using PostgreSQL and the
same API so all existing tools will work. Maybe not worth the effort or
maybe it will rock.

feedvalidator can generate a lot of useful information, but its plain text
output is an eyesore.  It should be able to generate an easier-to-read HTML
file as output.

Currently, if no service periods are defined, the feedvalidator doesn't
complain about that.

Transxchange2GoogleTransitCalendatDates.endElement():
Do not create calendar date exception when there are no
<DaysOfNonOperation> in <SpecialDaysOperation>

Also, the text of the issue description should be properly spaced so that
it doesn't run together when displayed in HTML.

A point in a shape that has the same distance value as a stop time in a
trip should be at roughly the same point in space.

When trying to execute schedule_viewer.py under Fedora, no matter if I
execute after running setup.py, it always report an import error:

Traceback (most recent call last):
  File "/usr/bin/schedule_viewer.py", line 26, in ?
    import marey_graph
ImportError: No module named marey_graph

What version of the product are you using? On what operating system?
I am using transitfeed-1.0.7. My operating system is version 2.6.20-1.2944.fc6

processing feeds takes tons of memory and it seems to get worse with each
release. Internally we have some large GTFS files that won't load on a 2GB
machine.

We need to do some profiling to work out if we are doing something that
prevents python doing the right thing.

It might be worth trying to reload a medium sized feed several times. If we
delete the Schedule object each time then the python process should stop
growing.

Here's someone with debugging a possibly similar problem
http://mail.python.org/pipermail/python-list/2007-June/445838.html
but he didn't get a reply.

Since Googles UI depends on the short route names being pretty short, maybe
flag a warning if the name is longer than, say, 6 chars?

If possible, flag a louder warning for 10+ chars?

In transxchange2GoogleTransithandler.writeOutput(): 
File names of individual feed files (archive.txt, routes.txt etc.) in
google_transit.zip are not consistent with Google Transit Feed
Specification; must not contain service start date as suffix in file name.

In GTFS stop times, the arrival and departure times should either both be 
specified or both be 
empty.  FeedValidator should check for this.  Also, there currently isn't a 
check for an arrival time 
that occurs after the departure time, and there should be.

Eclipse 3.2.2 on Linux (Distribution: Debian 3.1) cannot read the workbench
as checked into the repository. The workbench is created in Eclipse 3.2.2
on Windows XP. 

We need to change the library to support the stop_code field that was
recently added to the official spec, and possibly add relevant validation.

Some GTFS users (such as Google Maps) will ignore shapes attached to a trip
if the stops aren't close enough to points on the shape.  It would be nice
to catch that condition in the validator and generate a warning.

It would be nice to group (and limit) unused stop messages, since feeds
that have unused stops often have *many* unused stops, to the point where
they drown out other high-level issues.

Today AddStopTimeObject uses stoptime.GetTimeSecs(). Instead it should
check that previous.departure_secs <= current.arrival_secs

We should avoid using repr() in validation output, since the unicode
designator will be confusing to users unaccustomed to Python's string
representation.

Even though test/testexamples.py sets PYTHONPATH='.' the examples may be
run with a different older version of transitfeed.py if it is installed on
the system in an egg. You can see the problem by editing an example script
to print sys.path. As a work-around I tried running the examples as
python -c'import
sys;sys.path.insert(0,".");exec("example/feedvalidator.py")' other args but
couldn't get the quoting to work in windows. cmd.exe is not my friend.

If the expiration date of the schedule in the feed is in the near future (say, 
30 days), 
FeedValidator should give a warning

The spec is not very clear and the validator doesn't check. I don't think
it is intentional. We need to fix both.

A stop-lookup file is currently part of the distribution of
TransXChange2GoogleTransit. Not good practice in general, and in particular
as stops change frequently on a UK-wide basis. 
Task: Accept NaPTAN-format stop lookup file from the command line

Currently, the transitfeed module uses a simplified representation of
stop_times information--currently, a single time and a stop.  The pickup &
dropoff info, shape distance, etc. aren't handled yet.  This is the last
major hole in transitfeed's coverage.

Currently, the warning text isn't clear when a row in calendar.txt doesn't
have service days (because all service is defined in calendar_dates.txt). 
The gist of it is that the row isn't needed.

The validation script does not work on OS X due to incompatibility of diff
with Linux 

Unknown columns in feed files are ignored by parsers (to allow for future
extension), but we've seen occasional problems due to mislabeled optional
columns.  Providing an informative warning in those cases would make them
easier to discover.

It doesn't warn when day columns contain value not in ('0', '1')

An agency sent us a file with 8 <sp> characters for start_date and a tab
for end_date. validator from trunk didn't complain.

Currently, the FeedValidator is lenient about the number of cells (that is, 
unquoted commas) in 
each file.  This issue seems to occur frequently in new feeds, so it should be 
marked as an error 
in the FeedValidator.

The sample feed Marcy sent Joe and I (off the list) included invalid UTF-8
which crashed the validator.

ERROR: 7 issues found
Traceback (most recent call last):
  File "/home/thecap/src/transitdatafeed-trunk/python/feedvalidator.py",
line 198, in ?
    output_file.write(problems.GetOutput(os.path.abspath(feed)))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u02c7' in
position 2541: ordinal not in range(128)

feedvalidator should complain if a feed has anything other than consistent
CR LF line endings or consistent LF line endings.

If you run schedule_viewer from a different directory you get an error such as 
IOError: [Errno 2] No such file or directory:
'schedule_viewer_files\\index.html'

Work around is to change into install directory and run it there.

I should fix it to make an absolute path relative to __file__

Now that the spec offers a proper stop_code field, the validator should
warn if it seems like most stop names contain a stop code number.

Currently FeedValidator throws an exception when given a feed whose files are 
encoded in UTF-
16.  It would be useful to display more informative output.

schedule_viewer should provide a way to browse service periods, display
them on a calendar and allow the user to select periods.

Currently the list of departure times is a little confusing because it
displays times in any service period instead of the service on a particular
day. 

http://docs.python.org/dist/dist.html

After that is done we can add our code to the python index, cheeseshop.

What steps will reproduce the problem?
1. create a Trip using Trip()   # no parameters
2. add the Trip object to a Schedule
3. call schedule.WriteGoogleTransitFeed
4. you'll get exceptions that trip.direction_id and trip.block_id aren't
defined.

Once this is fixed, we can remove the gratuitous definitions of those
attributes from the TransitFeed sample code.

From the group:

"Greetings.  I wish I were able to suggest a fix however I can find a
possible bug.  My file validated (I ran it twice) and I pushed through
to FF.  He reported that there were too many columns and indeed there
were some ,,,, that were not returned as an error.  I can try to
recreate the datafile, if helpful, but I think you can get my drift."

When a quoted value is preceded by an extra space, it isn't treated as quoted:

value1, "value2, more", value3

is parsed into 4 values.

However, without the space, the value is parsed correctly:

value1,"value2, more", value3

This is coming from the python csv libary; we need to see if there's a 
workaround or whether we 
need to write our own CSV parsing to get around this.

schedule_viewer is really cool, so we should make it easier for people to
run by creating a Windows executable version.

feeds should never contain html or html escape entities (& for example).

It would be nice to group together (and limit) any issues that occurred many 
times in a certain 
run, in the same way that UnusedStop warnings currently are.  This would be 
handy because 
often a single change in the feed writer will fix all such errors for a feed.  
If we do this, we should 
also have a way to show all instances, in case the number of issues is slightly 
over the cutoff and 
the user wants to see all of them.

According to the spec, if routes.txt includes an agency_id column all rows
must have a value. Ideally Route.GetFieldValuesTuple would use the
schedule's default route if self.route_id is None and there would be a way
to totally omit the column if only one agency is in the schedule.
The easiest fix is to make agency_id required when creating a Route object
and populate it with the default agency in the loader when loading
routes.txt. I don't like adding things so that transitfeed loading and
writing a feed modifies its content.

Trips marked with a service_id that is non-existent in calendar.txt aren't
flagged as errors.