Comments (6)
Thank you for the report. Looks like the library needs to:
- Check if there are more than one channels returned by
Sensor.get_data()
- For a single channel, it is only channel
a
- For two channels, we have
a
andb
and need separateTHINGSPEAK_PRIMARY_ID_READ_KEY
instances
- For a single channel, it is only channel
- Abstract the channel parsing logic shared by both channels to their own functions
- Abstract each channel into its own class, i.e.
- Move
get_historical()
from Sensor to channel class, since a sensor is a representation of multiple channels
The design of this architecture is rather interesting and requires some additional thought. This will likely break compatibility with prior versions as well.
Notes:
Sensor channel data structure:
{
"mapVersion": "0.19",
"baseVersion": "7",
"mapVersionString": "",
"results": [
{
"ID": 2892,
"Label": "Imola - 1",
"DEVICE_LOCATIONTYPE": "outside",
"THINGSPEAK_PRIMARY_ID": "317405",
"THINGSPEAK_PRIMARY_ID_READ_KEY": "50UQJX9NR475QBWH",
"THINGSPEAK_SECONDARY_ID": "317406",
"THINGSPEAK_SECONDARY_ID_READ_KEY": "SOE9YW475VW6WZLA",
"Lat": 38.280572,
"Lon": -122.273712,
"PM2_5Value": "5.69",
"LastSeen": 1571048794,
"Type": "PMS5003+PMS5003+BME280",
"Hidden": "false",
"DEVICE_BRIGHTNESS": "15",
"DEVICE_HARDWAREDISCOVERED": "2.0+BME280+PMSX003-B+PMSX003-A",
"Version": "4.10",
"LastUpdateCheck": 1571047712,
"Created": 1502745962,
"Uptime": "3511208",
"RSSI": "-79",
"Adc": "0.0",
"p_0_3_um": "728.19",
"p_0_5_um": "200.53",
"p_1_0_um": "26.16",
"p_2_5_um": "2.89",
"p_5_0_um": "0.6",
"p_10_0_um": "0.0",
"pm1_0_cf_1": "3.99",
"pm2_5_cf_1": "5.69",
"pm10_0_cf_1": "6.69",
"pm1_0_atm": "3.99",
"pm2_5_atm": "5.69",
"pm10_0_atm": "6.69",
"isOwner": 0,
"humidity": "54",
"temp_f": "55",
"pressure": "1008.76",
"AGE": 487182,
"Stats": "{\"v\":5.69,\"v1\":5.53,\"v2\":5.62,\"v3\":5.64,\"v4\":6.64,\"v5\":6.08,\"v6\":3.31,\"pm\":5.69,\"lastModified\":1571048794041,\"timeSinceModified\":119982}"
},
{
"ID": 2893,
"ParentID": 2892,
"Label": "Imola - 1 B",
"THINGSPEAK_PRIMARY_ID": "317407",
"THINGSPEAK_PRIMARY_ID_READ_KEY": "V0FJ8SKLX6BOLB26",
"THINGSPEAK_SECONDARY_ID": "317408",
"THINGSPEAK_SECONDARY_ID_READ_KEY": "AYUATIO1HW0BIWR5",
"PM2_5Value": "5.96",
"LastSeen": 1571048794,
"Hidden": "false",
"Created": 1502745962,
"Adc": "0.01",
"p_0_3_um": "901.52",
"p_0_5_um": "263.74",
"p_1_0_um": "36.33",
"p_2_5_um": "0.91",
"p_5_0_um": "0.2",
"p_10_0_um": "0.0",
"pm1_0_cf_1": "4.39",
"pm2_5_cf_1": "5.96",
"pm10_0_cf_1": "6.01",
"pm1_0_atm": "4.39",
"pm2_5_atm": "5.96",
"pm10_0_atm": "6.01",
"isOwner": 0,
"AGE": 487182,
"Stats": "{…truncated...}"
}
]
}
Organization of sensor data:
- Sensor
- Parent
- Channel A
- ThingSpeak Primary
- ThingSpeak Secondary
- Channel B
- ThingSpeak Primary
- ThingSpeak Secondary
- Channel A
- Child
- Channel A
- ThingSpeak Primary
- ThingSpeak Secondary
- Channel B
- ThingSpeak Primary
- ThingSpeak Secondary
- Channel A
- Parent
from purple_air_api.
One problem with this approach is that the PurpleAir root API returns sensor channels as a flat list, so we need to do some pre-processing to get the root response to look the same as the one returned by the sensor API.
Channel a
data is returned as parent
sensors (i.e., without a parent
key) while channel b
data is returned as child
sensors (i.e. with a parent
key referencing the parent's ID). Since the API returns a list instead of a JSON blob, we need to build a hashmap of {ID: Channel Data}
so we can do lookups on that field.
from purple_air_api.
Started a branch: https://github.com/ReagentX/purple_air_api/compare/feat/cs/new-channel-arch
The abstractions are done, I just need to update tests, docs, and readme.
from purple_air_api.
Thanks again for the report, the above PR should address the issues.
from purple_air_api.
This release is now available in PyPI as version 1.1.
Note that the Blank
field does appear to have some data:
>>> from purpleair.sensor import Sensor
>>> se = Sensor('2890')
>>> df = se.parent.get_historical(weeks_to_get=1,
... sensor_channel='b',
... thingspeak_field='primary')
>>> df['Blank'].value_counts()
100.0 3166
98.0 162
99.0 99
97.0 87
96.0 66
95.0 65
49.0 61
61.0 60
46.0 54
45.0 54
60.0 51
47.0 45
48.0 39
56.0 35
59.0 34
55.0 32
92.0 31
62.0 31
57.0 31
81.0 31
52.0 30
78.0 30
50.0 29
66.0 29
76.0 28
73.0 27
69.0 27
79.0 27
70.0 26
94.0 26
67.0 26
71.0 25
80.0 24
84.0 24
82.0 23
53.0 23
91.0 23
51.0 22
77.0 22
58.0 21
68.0 21
64.0 21
72.0 20
54.0 19
90.0 19
88.0 19
65.0 18
93.0 18
74.0 18
75.0 18
87.0 18
83.0 17
63.0 17
89.0 16
85.0 14
86.0 13
44.0 2
Name: Blank, dtype: int64
If there are any new problems please open a new issue.
from purple_air_api.
I was wrong about the above, the field is truly blank:
created_at PM1.0_CF_ATM_ug/m3 PM2.5_CF_ATM_ug/m3 PM10.0_CF_ATM_ug/m3 UptimeMinutes RSSI_dbm Pressure_hpa Blank PM2.5_CF_1_ug/m3
entry_id
1002561 2020-09-09 00:01:09+00:00 1.03 1.41 1.41 18136.0 0.01 982.25 NaN 1.41
1002562 2020-09-09 00:03:09+00:00 1.07 1.60 1.60 18136.0 0.01 982.18 NaN 1.60
1002563 2020-09-09 00:05:09+00:00 1.28 1.59 1.76 18136.0 0.01 982.10 NaN 1.59
1002564 2020-09-09 00:07:09+00:00 1.33 1.71 1.71 18136.0 0.01 982.21 NaN 1.71
1002565 2020-09-09 00:09:09+00:00 1.25 1.86 1.86 18136.0 0.01 982.18 NaN 1.86
This bug will be fixed in the 1.1.1 release.
from purple_air_api.
Related Issues (20)
- Possible bug with `get_historical()` HOT 3
- ValueError: Invalid JSON data returned from network! HOT 1
- Discontinuation of the /json and /data.json urls
- `as_dict` and `as_flat_dict` should be channel methods, not sensor methods
- Organize FAQ
- Bug converting last_seen to datetime HOT 4
- RuntimeError: Polyfit sanity test emitted a warning, most likely due to using a buggy Accelerate backend. HOT 1
- Error in retrieving all sensor list HOT 5
- Deprecated API Endpoint (Invalid JSON data returned from network!) HOT 23
- Missing Parent Problem HOT 2
- Custom Time Range for get_historical HOT 1
- Add/Register DOI Badge HOT 1
- ThingSpeak field data type changed
- Field names disagreement with the PurpleAir documentation HOT 1
- Requests_cache HOT 4
- Help me in downloading the data by subsetting spatially and temporally
- Example code is out of date/wrong
- Parent sensors constructed from Child IDs have incorrect identifiers
- Limit to Historical Data HOT 2
- Formalize `thingspeak_args` parameters
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from purple_air_api.