udger / udger-php Goto Github PK

PHP agent string parser based on Udger https://udger.com/products/local_parser

License: MIT License

PHP 100.00%

bot-detection device-detector mobile-detection user-agent-parser

udger-php's Introduction

Udger client for PHP (data ver. 3)

Local parser is very fast and accurate useragent string detection solution. Enables developers to locally install and integrate a highly-scalable product. We provide the detection of the devices (personal computer, tablet, Smart TV, Game console etc.), operating system, client SW type (browser, e-mail client etc.) and devices market name (example: Sony Xperia Tablet S, Nokia Lumia 820 etc.). It also provides information about IP addresses (Public proxies, VPN services, Tor exit nodes, Fake crawlers, Web scrapers, Datacenter name .. etc.)

Tested with more the 50.000 unique user agents.
Up to date data provided by https://udger.com/

Requirements

php >= 5.5.0
ext-sqlite3 (http://php.net/manual/en/book.sqlite3.php)
datafile v3 (udgerdb_v3.dat) from https://data.udger.com/

Features

Fast
LRU cache
Released under the MIT

Install

composer require udger/udger-php

Usage

You should review the included examples (parse.php, account.php)

Here's a quick example:

Running tests

./vendor/bin/codecept run

Automatic updates download

for autoupdate data use Udger data updater (https://udger.com/support/documentation/?doc=62)

Documentation for programmers

https://udger.com/pub/documentation/parser/PHP/html/

Author

The Udger.com Team ([email protected])

old v2 format

If you still use the previous format of the db (v2), please see the branch old_format_v2

udger-php's People

Contributors

Stargazers

Watchers

Forkers

shawnbayer ratchet-stpup fabiomaggio adaamz crazyrex peshkov3 prospective codingfoundry sc-networks timeone-group dsentker24 moknomo flexmail seanpm2001

udger-php's Issues

SQLite fatal

PHP Warning: SQLite3::query(): Unable to prepare statement: 1, near "Tab10Q11": syntax error in ..udger/udger-php/src/Parser.php on line 43

Extend tests coverage

Add codeception or phpunit tests, and Travis integration.

Error: Call to a member function fetchArray() on boolean

PHP Warning: SQLite3::query(): Unable to prepare statement: 1, near "S": syntax error in vendor/udger/udger-php/src/Parser.php on line 252
PHP Fatal error: Call to a member function fetchArray() on boolean in vendor/udger/udger-php/src/Parser.php on line 254

[Symfony\Component\Debug\Exception\FatalErrorException]
Error: Call to a member function fetchArray() on boolean

Update documentation

example usage
list SQLite3 pecl package as dependency
how to download the data file

PSR logger

Echoing from the parser class in dangerous.
And the class should done one thing, and do it well, parsing agents.
Adding a PSR comptible logger would make the code more safe and easier to debug in large projects.

https://github.com/Seldaek/monolog

parser v3

Meta ticket for TODOs to finalize the parser for the v3 .dat files.

@mallat could you extend the TODO list here? I would offer dev support.

Create a release with guzzlehttp/guzzle ^7.0

For PHP8 compatiblity guzzle 7 is required. As stated in the release notes, guzzle 7 is supported, but this change did not make it into the latest release 3.1.4. The ^7.0 constraint is in master only.

Undefined variable htmlNL

PHP notice when running from the CLI

PHP Notice:  Undefined variable: htmlNL in /home/tiborb/Development/udger-php/udger.php on line 831
2016-01-21 16:36:51.239178      Setting cache dir to /tmp/udgercache/..
PHP Notice:  Undefined variable: htmlNL in /home/tiborb/Development/udger-php/udger.php on line 831
2016-01-21 16:36:51.239332      Setting Parse Fragments to 1..
PHP Notice:  Undefined variable: htmlNL in /home/tiborb/Development/udger-php/udger.php on line 831
2016-01-21 16:36:51.239375      Open DB file: /tmp/udgercache/udgerdb.dat..
PHP Notice:  Undefined variable: htmlNL in /home/tiborb/Development/udger-php/udger.php on line 831

single ticks in user agent string

the root cause of this issue was fixed with 2899965

Please create a bugfix release with this change. We stumbled upon the same problem with the string:

Mozilla/5.0 (Linux; Android 4.4.2; eZee'Tab10Q11-M Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Safari/537.36

so this is a real-world problem ;-)

Undefined offset

Hello!
Using Udger-php 1.4 installed via Composer in a Laravel 5 project. I am using the sqlite test data (free account) to test this out in our project.

When I enable the Fragments Parser I get undefined offset:

Code:

            // Setup Udger
            $uaparser = new Parser(true);
            // set data dir (this php script must right write to cache dir)
            $uaparser->setDataDir('/var/www/laravel/storage/udger');
            //If you want information about fragments
            $uaparser->setParseFragments(true);
            // Get UA Info
            $uainfo = $uaparser->parse($vdata['userAgent']);

Result:

2016-02-16 22:24:25.975610  Setting cache dir to /var/www/laravel/storage/udger..
2016-02-16 22:24:25.975746  Setting Parse Fragments to 1..
2016-02-16 22:24:25.975811  Open DB file: /var/www/laravel/storage/udger/udgerdb.dat..
2016-02-16 22:24:25.975863  Auto update is disabled, use existing db..
2016-02-16 22:24:25.976042  parse: start (useragent:Mozilla/5.0 (Linux; Android 5.1.1; SM-G925I Build/LMY47X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.95 Mobile Safari/537.36)..
2016-02-16 22:24:25.976113  parse: bot..
2016-02-16 22:24:25.976517  parse: browser..
2016-02-16 22:24:25.977706  parse: browser found (id: 423)..
2016-02-16 22:24:25.977971  parse: os..
2016-02-16 22:24:25.979139  parse: os found (id: 62)..
2016-02-16 22:24:25.979283  parse: device..
2016-02-16 22:24:25.980872  parse: device set by ua type - Mobile Browser..
2016-02-16 22:24:25.980941  parse: uptodate..
2016-02-16 22:24:25.981055  parse: fragments..
2016-02-16 22:24:25.981147  parse: get fragments..
2016-02-16 22:24:25.981391  parse: fragments parse..

  [ErrorException]     
  Undefined offset: 9

When I set ParseFragments to false it works fine.

Any assistance is appreciated. Thanks.

Add update data command

Downloading the current data file must be done manually at the moment.
Adding an update command would be really helpful.

It should:

check the validity of the access key
check the installed version
install a newer version if available at udger.com
the data file path should be configurable

Cheers,
Tibor

Update Guzzle version to 7

Notices in Parser.php

php full_example.php

PHP Notice:  Undefined variable: botName in /home/tiborb/Development/udger/udger-php/src/Parser.php on line 219
PHP Notice:  Undefined variable: botURL in /home/tiborb/Development/udger/udger-php/src/Parser.php on line 220

and sometimes from the parseFragments method

SQL parameterization breaks for several device codes

The udger_devicename_list table contains several entries with an unescaped ' in their code, so constructing the query on Parser.php:438 results in invalid SQL. Some examples:

INSERT INTO "udger_devicename_list" VALUES(1,33,'9.4'' FFF','9.4'' FFF');
INSERT INTO "udger_devicename_list" VALUES(1,15,'ALCATEL OT-5020A M''POP','One Touch M''Pop');
INSERT INTO "udger_devicename_list" VALUES(1,453,'eZee''Tab1001','eZee''Tab 1001');

I don't know if you'd prefer to solve this by removing ' as seems to be done for codes in other tables, or by preparing and parameterizing the query. (Or maybe both.)

PHP Notice in Parser

When the agent version was not found in:

                $info["ua_name"]          = $r["name"]." ".$result[1];
                $info["ua_ver"]           = $result[1];

it produces:

PHP Notice:  Undefined offset: 1 in udger/udger-php/src/Parser.php on line 323

Composer file and Packagist

Adding a https://getcomposer.org/ file would make it more easy to manage the parser in a project.
It could be published to https://packagist.org/ too.

Replace SQLite with PDO

Replace new \SQLite3 with a more generic driver: PDO.
http://php.net/manual/en/ref.pdo-sqlite.php

PDO is always available and no more composer check is needed.

node.js version

Udger could use a node.js version as well. Any thoughts on adding one?

Add a proxy option

Add a proxy option when performing a datefile update.

http://www.higherpass.com/php/tutorials/Using-Curl-To-Query-Remote-Servers/3/

Symfony bundle

Now that udger-php has a composer file, a symfony bundle could be created as well.

I could:

manage configuration of the access keys and serveral flags
offer caching
etc

Configurable API url

Some installations don't have direct internet access
and can't reach the udger api url hardcoded in src/Parser.php when performing an update (automatic at lookups or manual by calling the udateData method)).

Make the API url configurable.

Vars to review:

$api_url
$base_url
$ver_filename
$data_filename
$md5_filename
$resources_url

array(4) {
  ["flag"]=>
  int(1)
  ["info"]=>
  array(20) {
    ["type"]=>
    string(7) "unknown"
    ["ua_name"]=>
    string(7) "unknown"
    ["ua_ver"]=>
    string(0) ""
    ["ua_family"]=>
    string(7) "unknown"
    ["ua_url"]=>
    string(7) "unknown"
    ["ua_company"]=>
    string(7) "unknown"
    ["ua_company_url"]=>
    string(7) "unknown"
    ["ua_icon"]=>
    string(11) "unknown.png"
    ["ua_engine"]=>
    string(3) "n/a"
    ["ua_udger_url"]=>
    string(0) ""
    ["os_name"]=>
    string(7) "unknown"
    ["os_family"]=>
    string(7) "unknown"
    ["os_url"]=>
    string(7) "unknown"
    ["os_company"]=>
    string(7) "unknown"
    ["os_company_url"]=>
    string(7) "unknown"
    ["os_icon"]=>
    string(11) "unknown.png"
    ["os_udger_url"]=>
    string(0) ""
    ["device_name"]=>
    string(17) "Personal computer"
    ["device_icon"]=>
    string(11) "desktop.png"
    ["device_udger_url"]=>
    string(76) "https://udger.com/resources/ua-list/device-detail?device=Personal%20computer"
  }
  ["fragments"]=>
  array(0) {
  }
  ["uptodate"]=>
  array(4) {
    ["controlled"]=>
    bool(false)
    ["is"]=>
    bool(false)
    ["ver"]=>
    string(0) ""
    ["url"]=>
    string(0) ""
  }
}

Version 1.4.0

A new tag (v1.4.0) could be added for the new update command to be available on packagist and in composer.

Replace fopen with curl for updates

Replace fopen with curl when downloading a datfile because of security reasons: http://phpsec.org/projects/phpsecinfo/tests/allow_url_fopen.html

See setContents function for more details.