zamojski / towercollector Goto Github PK

I would like to ask you, is it possible for you to record the CellID without GPS function, or to insert in the settings a "Recording the CellID without GPS"?
Of course, the upload to OpenCellID must have to be blocked, but saving with empty GPS data should still be possible.

The background is that I'm looking for a very battery-saving tracking system, but this must not be accurate.

Another use case could be to map underground stations network coverage where GPS signal is not available but location is well known.

The Advanced setting page is all messed up. Example:

Export Database
    Export internal preferences

Export Preferences
    Export Database

Are you sure the json files produced are valid?

garbage after JSON object, at character offset 152475
(before ""items":[{"timestamp...") at /usr/bin/json_pp line 40.

https://stackoverflow.com/questions/38088384/can-a-json-file-contain-more-than-one-object
https://forums.pentaho.com/threads/136651-How-to-process-a-JSON-file-with-multiple-records/
OK, at least put a newline after each record.
Here is the workaround I used to create several json files.

perl -pwle 's/{"items":/\n$&/g' 2018-04-24-17-29-12.json |
sed /^$/d | split --lines=1 --additional-suffix=.json

I always export my measurements to CSV and GPX files before upload. Reasons for that:

• I like to have collected data archived
• errors can happen during upload
• OpenCellID database does may not store all columns
• API discards measurements on its own rules
• website publishes new measuremens only daily

Sometimes I need to have data available on the same day it has been measured, I use exported files. Waiting for export done is a time-consuming task after a long day when I have thousands on measurements in an old device.

I would like an option to create these backup files before upload. If file operation throws an exception (card full) upload should not be performed.

I use several devices for automatic data collection (see #1). It's not easy to stop recording on all devices when recording is not needed in the following situations:

• not moving for a few minutes or even hours
• being indoors
• arriving home

Sleep feature would help to:

• save battery
• save storage
• do not fill database with noise

My use case: I arrive home at late evening and my eyes are too tired to deal with a lot of phones to stop collecting. If application could go to sleep when idle I could forget all phones for night.

GPS is not suitable for movement detection as it may report false movements when signal is poor. Keeping GPS on would not help saving battery.

Accelerometer could help detecting idle time. If there is no movement for some minutes GPS may be switched off. Cell changes may be recorded with the remembered position. When accelerometer reports movement, collection should wake up and get GPS signal.

Another use case: friends could collect data in the background without any interaction without unnecessary battery drain (see also start collecting automatically on boot #1 and automatic upload #2).

Currently I alternate between TowerCollector and Mozilla Stumbler, so I can build up both data sets. Both OpenCellID and Mozilla location service accept WiFi information, which greatly increases the accuracy of location requests. However, I can only use Mozilla Stumbler to record WiFi data. If TowerCollector collected WiFi data, I could switch to only use TowerCollector, and upload to both databases from a single recording.

Please add support for icon shape
https://developer.android.com/guide/practices/ui_guidelines/icon_design_adaptive.html
https://material.io/guidelines/style/icons.html

you should fix it to upload new version to google play
https://www.xda-developers.com/google-play-app-updates-must-target-android-8-0/

This is a bad design:

The good design is to block (greyout) upload button to show user it's disabled.

You can leave popup explanation to clarify why it's exactly blocked.

Hello.

This is a common crosspost for opencellid, radiocells and mozilla location.

The rapid transit (metro,subway, tube) in my city has a very good GSM, WCDMA and LTE coverage. On the stations and inside tunnels. We have 4 different real (not virtual) operators so there a lot of stations. And there are no GPS or any other GNSS coverage, of couse.

So I can’t fill the database with TowerCollector Radiobeacon or Mozilla Stumbler.

But it will be very-very coll to see on the map where you are during the trip.
Moreover, I’m dreaming about the app that will notify me when should I exit (the station before).

I’m ready to fill the database manually: to visit all stations, make records, and publish them associating according to each station.

There is also wifi in each train (wagon), but I see no reason to put it into database as it’s moving.

http://wiki.opencellid.org/wiki/Main_Page

I just tried to export data I've collected today during an excursion. This was the first time I used this functionality on that device. TC triggered the permission dialog, I granted (looking into Apps › Tower Collector › Permissions confirms this) – and got a toast informing me "Export failed, directory could not be created".

Unfortunately, there's no setting where I could specify a directory. Neither does it tell me where it wanted to create one, so I could not do that by hand.

So where to look for and create it – and then, how to fix this issue?

LG P880 running Android 6.0.1 (inofficial LOS). TC installed from the Github releases (via my repo).

The latest release (tag: 1.14.0) unfortunately lacks the .apk file (probably forgotten). Could you please attach it? Thanks in advance!

If enabled to collect neighboring cells display information if the feature is supported by the device.

Display number of neighboring cells for last measurement.

Provide option to contribute to Mozilla Location Services through upload and JSON file.

The upload arrow, present at all times, should be grayed out when there is nothing to upload.
Also it should be animated, during uploads.

For 3G networks the Last Saved window shows both the cell ID and the long cell ID. However for 4G LTE networks only the long CellID is shown. Also it is just called CellID.

Make CSV file columns compatible with OpenCellID.org API so the file can be uploaded later through website without conversion.

OpenCellID is going to shut down (if no one will pay for it), see announcement at http://wiki.opencellid.org/.

Obvious solution for the future would be to contribute to Mozilla Location Service instead.

Do you agree?

I have asked a friend for collecting data on train lines where he works. Some weeks later I have checked data but no measurements were in the database. He made sure collecting was started, but did not know that upload also needs to be started manually.

A background service could check if conditions are suitable for upload:

• automatic upload is enabled (with option to check Android sync option)
• WiFi is available (or mobile data if a setting is enabled for that)
• collecting is not active (stopped or GPS is off or paused by automatic sleep)
• there is enough data or time for upload (configurable)

I think it's confusing that a triangle (play) button serves for recording data. A circle (record) button would be easier to understand.

Display confirmation message before performing upload.

Apply theme without restarting the app, now you need to restart the app to change color theme.

PCI is correctly named in CSV, but lumped with PSC in JSON.

Currently it's possible to save collected data to GPX. Would be nice to also directly open the exported file with another application, e.g. Location Map Viewer, to display the track on a map. LMV even accepts GPX data via intent, but also seems to have registered for the file type: I used a file manager to navigate to the exported .gpx file, and tapping it opened LMV. Other apps might offer the same.

Provide option to inform external apps installed on the system that new measurements have been collected. This would allow other application to use Tower Collector data if interested.

For those using this application every day, being able to disable notifications in the notification tray could help with de-cluttering their phone.

The behavior of TowerCollector is interesting if airplane mode (cellphone off) is turned on while TowerCollector is collecting data. One may do this due to wanting not to be disturbed. Once airplane mode is turned on (cellphone on) TowerCollector seemed confused.

If I change the network type for eg. from LTE --> 3G while TowerCollector is scanning, it doesn't seems to detect the change. I still see in the stats that only one LTE tower is detected and nothing else.

Once I stop the scanning, and upload the detected LTE cell tower, and then switch on the scanning again, now the app detects the new network type and shows me the new UMTS tower detected. Quite cumbersome!

I'm running your app on Oneplus One, android 7.1.1.

EDIT: It seems that stopping and uploading is not necessary; once I went outside and started moving the new network type got detected.

As a dedicated option in the menu add the possibility to clean all gathered information for privacy reason.

Stroll throgh your favorite giant empty parking lots and wide open fields.
Playing with GPS apps, they say 3 meters accuracy.
Sure, there are some 5.36 m accuracy points, but all the points recorded by TowerCollector in those parking lots are 17, etc. m.
Anyway, there seems some other factors involved.

$ perl -nwle 'print for /GPS accuracy ([\d.]+) m/g;' 2018-09-17-06-39-41.gpx|sort -n|uniq -c|sort -k 2n
      7 5.36
      6 6.43
     11 7.5
      8 8.58
     15 9.65
     16 10.72
     18 11.79
     15 12.86
     17 13.94
     15 15.01
     27 16.08
     21 17.15
     20 18.22
     23 19.3
     17 20.37
     11 21.44
      9 22.51
      8 23.58
      6 24.66
      7 25.73
      8 26.8
      5 27.87
      6 28.94
     10 30.02
     11 31.09
      3 32.16
      3 33.23
      1 34.3
      3 35.38
      4 36.45
      7 37.52
      8 38.59
      7 39.66

I don't if it is possible, but what if the app could rotate the network type automatically i.e. 2G --> 3G --> 4G and back to 2G with some preselected interval and thus make the collection more efficient. Then users would not need to go the same routes over and over again after manually selecting different network type.

Could you make the app to switch wifi on on new session start.

If GPS signal is lost and TowerCollector is running it appears TowerCollector starts to eat the battery. I'm guessing TowerCollector continuously tries to regain location information. Checking frequently immediately after the GPS signal has been lost is okay, but once signal has been lost for a period TowerCollector should back off and check less frequently to save battery. Most likely if minimal motion is seen, the person/phone is inside a building; larger motion might suggest a tunnel.

Exported files can be large and have repetitive content. Simple gzip compression can save lots of card space.

I have checked an average amount of measurements with about 3500 rows:

• CSV: 500k uncompressed, 100k compressed
• GPX: 1500k uncompressed, 120k compressed

Compression option could be a checkbox on export dialog that saves its state into preferences.

Provide option to define areas from which cells will be discarded while collecting. Basically this would allow the user to specify an area (by defining center and radius) where he doesn't want the app to scan and pick up cells. Starting the app when leaving home results in lots of readings around.

Of the two CSV export formats,

• There is nothing in the files that says which is which.
• There is nothing in the filenames that says which is which.

The only way to tell which is which is to

1. Note which of the export buttons you pressed first.
2. Note the order of the times of the filenames produced.
3. Write down on a piece of paper what differes in the innards of each of the two files produced.

So I propose that instead of just *.csv, they should be written to *.x.csv. and *.y.csv, or better letters, and the letter should be there in the export menu too.

Why you should switch:

The Mozilla Public License (MPL) was originally created by Mozilla, but its GPL-incompatibility caused so many problems that Mozilla eventually re-licensed their work under a GPL/LGPL/MPL triple license. It's important to note that even the original creator of the MPL 1.0 and 1.1 abandoned it as their sole license, because of GPL-incompatibility; don’t duplicate their mistake. Google no longer lists the MPL as a recommended license, even though they once did.

The following clause means that this license will (and may already) have conflicts with other open source licenses:

1.5. "Incompatible With Secondary Licenses"
    means

    (a) that the initial Contributor has attached the notice described
        in Exhibit B to the Covered Software; or

    (b) that the Covered Software was made available under the terms of
        version 1.1 or earlier of the License, but not also under the
        terms of a Secondary License.

Hence, we should switch to the GNU GPL as it is the most widely used free software license and has a strong copyleft requirement whereas the Mozilla Public License 2.0 is seen as a "weak copyleft". Additionally, as previously stated, it will prevent further issues of incompatibility for anyone who wants to fork this project.

All that needs to be done is change the license to the GPL in this repo and it will affect all further versions. Simple as that.

If I've understood correctly, this app seems to be open source. If so, would be nice to see it added into F-Droid (open source app "store"), since majority of privacy minded people don't want to install GAPPS on their phone and thus don't have Play Store access. Currently there isn't a single app on F-Droid that collects cell towers for opencellid database, and this could be the one!

https://f-droid.org/

Hi,

I would like to translate this app in French to help people join this project.
But there are a lot of files there. Which ones should I translate exactly (other than the strings.xml file)?

Pressing the export choices mentioned in #38 result in a Unsafe Operation warning.

All we are doing is writing an export file, not importing one.

Provide option to present selected types of networks on the map.

I hate to say it but the Upload Button is so close to the ON/OFF button,
and also the menu, that the user is very likely, sooner or later, to
accidentally touch the Upload Button, sending his precious data to
upstream, and deleted from his device, before say, he has a chance to
save it to GPX etc. first, whatever.

In fact one might say to only safe way to use the app is when in
Airplane Mode, when an accidental tap of the Upload Button will fail.

One might say "who cares, your data is now upstream." But in fact it is
very hard to retrieve ones data back from the two current upstream
locations.

Therefore perhaps there should be an option:

[ ] Ask for confirmation when hitting the Upload Button

Curious if any of the formats lost data, I tried

$ grep -c 466.*92 * #MCC MNC
2018-09-17-06-39-41.gpx:353
2018-09-17-06-39-49.csv:353
2018-09-17-06-39-35.csv:353
2018-09-17-06-39-55.json:323

Why did json lack some points?

One reason seems to be during a cell transition when stationary, in the
json file the new cell is ignored:

2018-09-17-06-39-49.csv:254:24.14220491,120.63880148,466,92,,,24950,,105859362,,,387,60,,"2018-09-16 08:32:21.225Z",28,2,120,LTE,"asus ASUS_Z012DA"
2018-09-17-06-39-49.csv:255:24.14220491,120.63880148,466,92,,,24950,,105859352,,,388,49,,"2018-09-16 08:32:24.269Z",28,2,120,LTE,"asus ASUS_Z012DA"
2018-09-17-06-39-55.json:74:{"timestamp":1537086741225,"position":{"latitude":24.14220491,"longitude":120.63880148,"accuracy":27.87,"altitude":84,"heading":120.1,"speed":2.24,"source":"gps"},"cellTowers":[{"radioType":"lte","mobileCountryCode":466,"mobileNetworkCode":92,"locationAreaCode":24950,"cellId":105859362,"primaryScramblingCode":387,"asu":60,"signalStrength":-80,"serving":1},{"radioType":"lte","mobileCountryCode":466,"mobileNetworkCode":92,"locationAreaCode":24950,"cellId":105859352,"primaryScramblingCode":388,"asu":49,"signalStrength":-91,"serving":1}]},

OK but what is more valid, the old cell or the new cell? And what if the
new cell only occurs once, thus not the case of 'who cares, there will
be plenty of it following later'. And even if in that case, a valuable
data point is thrown away.

Anyway one hopes that within the app one can find a warning that JSON
contains less data points.

I have several Android devices that I use exclusively for automatic surveying. Two devices (for GSM and UMTS) per operator, six phones. I switch them on before leaving home. It is a very time-consuming task to wait for boot, then wait for Tower Collector to open and then start collecting.

If Tower Collector would be configurable to start collecting on startup, I had only to switch devices on. The same option could also help non-geek people to collect data continuously.

Of the four output formats,
three are consistently ordered by timestamp.
The JSON format is in crazy order.
Please order it by timestamp too.

The EARFCN is available from the system but not logged anywhere.

Please show the Cell ID in the format all the other apps do, e.g., 29769971 = 00454 16627. In fact showing both formats at the same time would be fine.

The cell id (an integer in the range of 0 to 268435455). This long cell id is constructed as 65536 * RNC + CellID (where the RNC means Radio Network Controller and the CellID is an integer in the range of 0 to 65535)

Allow me to save my measurements locally, even after uploading to OpenCellid. I would like to have access to all my measurements - I don't like that they get deleted. OpenCellid has been having server problems and I'm unable to down my measurements from their site.

Display information if the feature is supported by the device.