adriangibbons / php-fit-file-analysis Goto Github PK

If anyone receives:

phpFITFileAnalysis->readDataRecords(): this class can only handle normal headers!

Please share an example FIT file and I'll use this for testing the handling of these timestamps. I'd be flying blind without a file to work so I'm not developing this until someone experiences the Exception above being thrown!

Adrian.

In the FIT files there are two fields "product" and "garmin_product" which are not interchangeable. Therefore, when pulling out the device information for Garmin devices some values that are returned are incorrect. The code does not distinguish between the "product" fields and the "garmin_product" fields in the FIT file. . I reached out to Garmin directly and they noted that "product" has something to do with Garmin Connect and does not reference the product list from the FIT SDK. I had thought I could simply line up the manufacturer array ( with "1" representing Garmin manufacturer), but this doesn't work either. I'll illustrate with an example. If I run the code on a FIT file I will get a ['device_info]['product] array with 8 entries. However, I will get a manufacturer array with only 7 entries. It seems some values of 'product' in the FIT file do not have a corresponding 'manufacturer' value (while all 'garmin_product" ones do). I manually went through the FIT file and pulled the values to illustrate this point:

manufacturer 1 garmin_product 2050
manufacturer 1 garmin_product 2050
product 558
manufacturer 1 garmin_product 1620
manufacturer 1 garmin_product 2050
manufacturer 1 garmin_product 2050
manufacturer 1 garmin_product 1620
manufacturer 25165 product 3

note no manufacturer for 'product' 558.

Therefore without knowing which values in the ['device_info']['product'] array correspond to 'product' or 'garmin_product' it is impossible as far as I can tell to line these arrays up.

Therefore, $this->data_mesgs['device_info']['product'] should NOT contain values pulled from the "product" field. Is there some small change to the code that can be made to only pull the "garmin_product" field and ignore the "product" field?

Hi,

All values of cadence filed are 0, when Stryd Power IQ app installed. Could you handle this problem? Thanks.

Stryd Power IQ app link:
https://apps.garmin.com/en-US/apps/660a581e-5301-460c-8f2f-034c8b6dc90f

The activity with Stryd Power IQ app:
activityWithStydPowerIQApp.fit.txt

Getting the following error when using the attached FIT file with HR:
_(1/1) FatalErrorExceptionUnsupported operand types
in phpFITFileAnalysis.php (line 2713)

Using the latest master version - looks like $last_event_timestamp is an array of 2 elements rather than the expected integer but not sure how you match the correct version to use..

error3.txt
Tried uploading to garmin connect and FIT file is working ok.

http://www.php-fig.org/psr/psr-4/

I'm trying to process a file that looks to be from a Suunto Ambit that I can view by uploading to https://www.fitfileviewer.com/ If I try to process using this library I get Undefined offset error @ $this->types = $this->endianness[$architecture];

$architecture is showing as 14. Obviously, the code expects zero or one. Any idea? Fit file attached.

68270818303.zip

Hi,

I have an issue with the parser.

( ! ) Notice: Undefined offset: 84 in \vendor\php-fit-file-analysis\phpFITFileAnalysis.php on line 1228

Do you have any idea about this issue ?
Many thanks

Hi

First, thanks for a truly useful project that is making me attempt an exercise database so much more doable.

Apologies, this is a question rather than an issue. Whilst most applications show lap splits, nothing really allows drilling down into what I term 'legs' i.e. portions of activity between pauses of the timer during a lap. I have a test file where the end of lap 2, the start of lap 3, and one location record all have the same UNIX timestamp and I'm not sure how to calculate which lap this record belongs to. Is it possible for me to iterate through all events (timer pauses), lap markers and location records together in chronological order so as to know which lap to assign a particular record to? Or perhaps access fractions of a second (assuming they exist) for each record?

fixData() just ensures timestamp consistency, probably need a function to ensure data is in there for every second of recording...

Great library.

How do you go about matching up the indexes when some are missing?
See attatched - its for the 'lap' element from a swim.
Some of the 'total_distance' elements are 0 (resting period) but this means the indexes are missing from 'swim_stroke' making it pretty hard to match up the corresponding distance and stroke. Is this is a bug or is that the way the data comes through? I looked at the 'fix data' option but it only works for certain elements.

Is it possible to have new getJSON functions for other FIT sections? This function extracts only data from "record" section, but I need also an extractor for "lap" section.
Thank you so much for this helpful lib!

Would be useful to get data in a JSON format e.g. for use in dc.js

{
    "fix_data": "all",
    "units": "metric",
    "pace": null,
    "data": [
        {
            "timestamp": 71234828,
            "lap": 1,
            "pos_latitude": 32.15,
            "pos_longitude": 115.24,
            "speed": 29.3,
            "altitude": 15,
            "cadence": 91,
            "heart_rate": 116,
            "power": 189,
            "chainring": 53,
            "sprocket": 19
        },
        {
            "timestamp": 71234832,
            "lap": 1,
            "pos_latitude": 32.16,
            "pos_longitude": 115.23,
            "speed": 28.1,
            "altitude": 15,
            "cadence": null,
            "heart_rate": 121,
            "power": 0,
            "chainring": 53,
            "sprocket": 18
        }
    ]
}

Could create the array structure in PHP and then use:

<?php json_encode(); ?>

I've just made some non-backward compatible changes and bumped the versioning to v3.0.*

I've updated the README - hopefully I haven't missed anything. PSR-4 autoloading to come...

The project now has its own namespace and is starting to be more PSR-2 code styling compliant! E.g. method names are now camelCase - power_metrics() has become powerMetrics() etc.

It's also available on packagist.org too, for easier updating. Just add to your project's composer.json file:

{
    "require": {
        "adriangibbons/php-fit-file-analysis": "3.0.*"
    }
}

e.g. start and end timestamps for each lap and session

Seeing a lot of 255 - which would appear to be invalid values at first glance...

I will suggest a litte change:

I have a REST api, that api have a method in order to import a FIT file. I am using this class after import, but, is imposible load RAW content. I need first write the content in a file and after that load with phpFITFileAnalysis.

Have sense a function like that?
loadFromFile($file_path, $options = null)
loadFromData($data, $options = null)

When importing a garmin 820 fit file it errors with "this class can only handle normal headers":

if (($record_header_byte >> 7) & 1) { // Check that it's a normal header
throw new \Exception('phpFITFileAnalysis->readDataRecords(): this class can only handle normal headers!');
}

Anyone has updated code?

Included a copy of the problem fit file.

niels probleem 820 2016-09-07-05-49-08.zip

Regards,
Henk

Forgive me if this is overly simplistic, however, I'm new to Github and not familiar with the way things typically work. This is more of a feature request...

The rounding of units is restrictive in some cases. For example, distance fields are rounded to 2 decimal places. In many cases, due to the restriction, the maximum data resolution is greater than the maximum GPS error at 95% confidence level. I would kindly suggest considering 3 place precision on distance fields, or perhaps a config option to allow the developer a little more flexibility.

I can edit my files to do what I need, but of course, I would prefer not to create my own custom branch of the code.

Thank you for your work on this useful resource, Adrian.

Cheers.

It's great that your code analyse fit files.
is the opposite diretion possible: Is there a possibility to create fit files starting from some coordinates ?

https://github.com/adriangibbons/php-fit-file-analysis/blob/master/src/phpFITFileAnalysis.php#L907-L909

    private function readDataRecords()
    {
        $record_header_byte;
        $message_type;
        $local_mesg_type;

I'll get error on those lines (with HHVM - not sure about normal PHP)

btw: thanks for all those changes, I just came from holiday and seen all the work you've done.

The first entries for position_lat and position_long in one of my FIT files are [1539031114] => -40.64575 and [1539031114] => 175.29212. Yes, I'm in New Zealand!

However, I have lap[start_position_lat] => 220.64569, lap[start_position_long] => 175.29196, lap[end_position_lat] => 220.86769, and lap[end_position_long] => 175.06788, and session[start_position_lat] => 220.64569, session[start_position_long] => 175.29196.

It looks as though the conversion for these lap[] and session[] fields isn't working correctly for negative latitudes. I need to subtract the lat values from 180 to get the correct -ve value. The conversion obviously works OK for the position_lat values.

Can anybody point to me to where this needs to be fixed (in function readDataRecords()) please??

Of course it may be my device, Lezyne Mega XL, but I also see the same problem with FIT files from a friend's device from a completely different manufacturer.

Thanks.

Fjeldfross

Hello,
would it be possible to add composer.json file and submit it to https://packagist.org for easier updates?

Hi, im testing your parser (great job) but i have a problem.

I have an issue with the parser. Unfortunatly, data_mesgs['session'] is null and i don't have any data.
data_mesgs['record'] is ok, full of data. the fit file is ok, is a bike workout (2 hours duration).
recorded using forerunner 935. i tested the fit in fitfileviewer.com and is ok.
do you know where is the problem?

thanks so much.

I just updated the version to v3.1 and there is an error of "Unsupported operand types"
in /home/ubuntu/workspace/vendor/adriangibbons/php-fit-file-analysis/src/phpFITFileAnalysis.php at line 1446 from TomTom user.

Could you please fix this or What should I do? Thank you.

run-20180322T212609.fit.TXT

Need to compare performance and output of phpFITFileAnalysis::sma() vs the PHP Trader PECL extension

When decoding a .fit file with your lib I don't get real timestamp.

From your lib : 1995-10-03
From Strava : 2015-10-01 (good one)
Timestamp : 812681141

Power, cadence, etc... are fine but I can't figure out why timestamp is wrong.

Do you have any clues ? I can send you the file if needed.

Likely to be following this commit f3379fc

http://www.php-fig.org/psr/psr-2/

Hi,

for the first time Garmin (yesterday and today) has sent me a FIT with multiple sessions, that generate an error.

How to approach this?

Thanks a lot,
Franco

Hi Adrian,

Thanks for the recent fix. In addition to that issue, there is another problem I've been encountering with a specific FIT file. Unfortunately, I am still quite unfamiliar with the FIT spec and cannot pinpoint what the problem is during parsing. What I can tell you is that it gives the error:

Undefined index: record

And that it works in GoldenCheetah. I have uploaded the file to https://www.mediafire.com/?hgc99xr1oc5ic98

Thanks.

Hi,
How does the code separate between a Pause or just a Smart Record (sampling not every second)?

Hi,

is there also a "mountain-biking.php" example using the OpenStreetMap services with leaflet.js instead of Google maps?

Regards

Hello Adrian :)

I have an issue with the parser. Unfortunatly, data_mesgs['record'] is null and i don't have any data.

This line seems to be the problem :
$this->file_header['data_size'] = $this->file_header['crc'] - $header_size + 2;

It's a session recorded with Garmin Forerunner 920XT.

Many thanks,

Hi Adrian,

I love your php class and use it for a personal project to show my cycling data.
Now I would like to put into mysql db all my fit files for a few years so I can create a custom reports and graph.

I would like to have your opinion on how to organize the mysql db tables.

Thanks.

Daniele

Hi again,

Run into the above error with a FIT file from a brand new Garmin Forerunner 395 watch. If you comment out the line that throws the error it will parse it but some information is missing.

Ie Record->heart_rate is not present (but it is present in the lap array). This information is definitely in the file as if you upload it to Garmin connect its shown.

This watch seems to have been added between FIT SDK versions 20.16 and 20.30 (latest) somewhere. Not sure if its stored in a different field in the fit file (as this watch uses a different way of capturing heart rate I belive)

1627845922.txt

New public function to take TSS's as input and return:

• Acute Training Load (Fatigue)
• Chronic Training Load (Fitness)
• Training Stress Balance (Form)

E.g. getFatigueFitnessForm($tss_values, $ctl_time_period = 84days, $atl_time_period = 14days)

Which may be related to how records are being parsed internally. The error being output is:

  [ErrorException]            
  Undefined index: timestamp  

I can provide FIT files on request.

The process of reading in the file: file_get_contents(), followed by str_split() and then array_reverse() uses a lot of memory and without memory_limit in php.ini set to a suitable value then _Fatal Error: Allowed Memory Size of xxxxxx Bytes Exhausted_ may be experienced. Some users may not have access to change the memory_limit therefore alternatives should be investigated.

For example, reading a FIT file of 263kb:

before file_get_contents(): 524,288 bytes
after file_get_contents(): 1,048,576 bytes
after str_split(): 28,835,840 bytes
after array_reverse(): 44,040,192 bytes (~44mb!)
(using memory_get_peak_usage(true) to return the peak of memory allocated by PHP.)

By the end of the script, approximately 78mb (peak) was used.

One possible solution would be to use substr() on the string read by file_get_contents() instead of splitting the string into an array, reversing it, and using array_pop().

Hi adriangibbons,
First, your fit file reader is great! Thanks for this.
However I have read a fit file created with Garmin Forerunner10 and the altitude information is missing. In my Garmin Connect account I can see the altitude. So I think that the altitude information is in the fit file.
I have also output the fild information - also no altitude exists. Could you check this issue please? Here is a downloadlink to my fit file: http://stephan.mein-sporttagebuch.de/userfiles/tempfiles/514B0111.FIT
Thank you very much in advance.
Kind regards,
Stephan

Raw values for lat and long of 2,147,483,647 resulting coordinates of 180,180 on the map.

Approx. 1% (24 of 2,368) data points affected from Forerunner 110.

session and lap messages can contain data that will need units setting - e.g. average and max speed etc.

Hi Adrian,
I am having a problem with speed values that I was hoping you could help me hunt down. I am getting a larger value for average speed than I am for max speed when I var_dump data_mesgs['lap'].
Below is what I am getting for those two sets. I have the fit file available here: https://www.dropbox.com/s/ep31ebzc7s1izu3/file.fit?dl=1
Thank you,
Alex

["avg_speed"]=>
array(8) {
[0]=> float(5.457)
[1]=> float(5.002)
[2]=> float(6.198)
[3]=> float(4.108)
[4]=> float(6.217)
[5]=> float(4.052)
[6]=> float(5.842)
[7]=> float(3.766)
}
["max_speed"]=>
array(8) {
[0]=> float(4.868)
[1]=> float(4.88)
[2]=> float(4.908)
[3]=> float(3.502)
[4]=> float(3.987)
[5]=> float(3.801)
[6]=> float(3.841)
[7]=> float(3.573)
}

Thank you good script.Sir. very very good.
I have GARMIN EDGE1000 and shimano Di2 D-fly.
But I cant get Di2 gear change and gear info data on GARMIN EDGE 1000 RAW fit file.
Is this script cant yet?
I trust this script can it, But I dont understand pick up gear data method, thanks.
If you need gear data in fit file. please message me. I gonna send it.
RGDS.

I am storing Garmin files into a s3 bucket. But when I try to fetch the file from s3 it is displaying file not found. This is the error which I am getting
phpFITFileAnalysis->__construct(): file 'https://s3-us-west-2.amazonaws.com/uploadedimg/garmin_upload/6465920425' does not exist! in [project_root]/vendor/adriangibbons/php-fit-file-analysis/src/phpFITFileAnalysis.php on line 944

Here is the code provided in my controller

$filepath = 'https://s3-us-west-2.amazonaws.com/uploadedimg/garmin_upload/6465920425';
$options = ['units' => 'statute'];
$pFFA = new \adriangibbons\phpFITFileAnalysis($filespath,$options);
$grCnt = $pFFA->data_mesgs['session'];