Git Product home page Git Product logo

Comments (6)

ReagentX avatar ReagentX commented on August 17, 2024 1

Thank you for the report. Looks like the library needs to:

  1. Check if there are more than one channels returned by Sensor.get_data()
    • For a single channel, it is only channel a
    • For two channels, we have a and b and need separate THINGSPEAK_PRIMARY_ID_READ_KEY instances
  2. Abstract the channel parsing logic shared by both channels to their own functions
  3. Abstract each channel into its own class, i.e.
  4. Move get_historical() from Sensor to channel class, since a sensor is a representation of multiple channels

The design of this architecture is rather interesting and requires some additional thought. This will likely break compatibility with prior versions as well.

Notes:

Sensor channel data structure:

{
    "mapVersion": "0.19",
    "baseVersion": "7",
    "mapVersionString": "",
    "results": [
        {
            "ID": 2892,
            "Label": "Imola - 1",
            "DEVICE_LOCATIONTYPE": "outside",
            "THINGSPEAK_PRIMARY_ID": "317405",
            "THINGSPEAK_PRIMARY_ID_READ_KEY": "50UQJX9NR475QBWH",
            "THINGSPEAK_SECONDARY_ID": "317406",
            "THINGSPEAK_SECONDARY_ID_READ_KEY": "SOE9YW475VW6WZLA",
            "Lat": 38.280572,
            "Lon": -122.273712,
            "PM2_5Value": "5.69",
            "LastSeen": 1571048794,
            "Type": "PMS5003+PMS5003+BME280",
            "Hidden": "false",
            "DEVICE_BRIGHTNESS": "15",
            "DEVICE_HARDWAREDISCOVERED": "2.0+BME280+PMSX003-B+PMSX003-A",
            "Version": "4.10",
            "LastUpdateCheck": 1571047712,
            "Created": 1502745962,
            "Uptime": "3511208",
            "RSSI": "-79",
            "Adc": "0.0",
            "p_0_3_um": "728.19",
            "p_0_5_um": "200.53",
            "p_1_0_um": "26.16",
            "p_2_5_um": "2.89",
            "p_5_0_um": "0.6",
            "p_10_0_um": "0.0",
            "pm1_0_cf_1": "3.99",
            "pm2_5_cf_1": "5.69",
            "pm10_0_cf_1": "6.69",
            "pm1_0_atm": "3.99",
            "pm2_5_atm": "5.69",
            "pm10_0_atm": "6.69",
            "isOwner": 0,
            "humidity": "54",
            "temp_f": "55",
            "pressure": "1008.76",
            "AGE": 487182,
            "Stats": "{\"v\":5.69,\"v1\":5.53,\"v2\":5.62,\"v3\":5.64,\"v4\":6.64,\"v5\":6.08,\"v6\":3.31,\"pm\":5.69,\"lastModified\":1571048794041,\"timeSinceModified\":119982}"
        },
        {
            "ID": 2893,
            "ParentID": 2892,
            "Label": "Imola - 1 B",
            "THINGSPEAK_PRIMARY_ID": "317407",
            "THINGSPEAK_PRIMARY_ID_READ_KEY": "V0FJ8SKLX6BOLB26",
            "THINGSPEAK_SECONDARY_ID": "317408",
            "THINGSPEAK_SECONDARY_ID_READ_KEY": "AYUATIO1HW0BIWR5",
            "PM2_5Value": "5.96",
            "LastSeen": 1571048794,
            "Hidden": "false",
            "Created": 1502745962,
            "Adc": "0.01",
            "p_0_3_um": "901.52",
            "p_0_5_um": "263.74",
            "p_1_0_um": "36.33",
            "p_2_5_um": "0.91",
            "p_5_0_um": "0.2",
            "p_10_0_um": "0.0",
            "pm1_0_cf_1": "4.39",
            "pm2_5_cf_1": "5.96",
            "pm10_0_cf_1": "6.01",
            "pm1_0_atm": "4.39",
            "pm2_5_atm": "5.96",
            "pm10_0_atm": "6.01",
            "isOwner": 0,
            "AGE": 487182,
            "Stats": "{…truncated...}"
        }
    ]
}

Organization of sensor data:

  • Sensor
    • Parent
      • Channel A
        • ThingSpeak Primary
        • ThingSpeak Secondary
      • Channel B
        • ThingSpeak Primary
        • ThingSpeak Secondary
    • Child
      • Channel A
        • ThingSpeak Primary
        • ThingSpeak Secondary
      • Channel B
        • ThingSpeak Primary
        • ThingSpeak Secondary

from purple_air_api.

ReagentX avatar ReagentX commented on August 17, 2024 1

One problem with this approach is that the PurpleAir root API returns sensor channels as a flat list, so we need to do some pre-processing to get the root response to look the same as the one returned by the sensor API.

Channel a data is returned as parent sensors (i.e., without a parent key) while channel b data is returned as child sensors (i.e. with a parent key referencing the parent's ID). Since the API returns a list instead of a JSON blob, we need to build a hashmap of {ID: Channel Data} so we can do lookups on that field.

from purple_air_api.

ReagentX avatar ReagentX commented on August 17, 2024 1

Started a branch: https://github.com/ReagentX/purple_air_api/compare/feat/cs/new-channel-arch

The abstractions are done, I just need to update tests, docs, and readme.

from purple_air_api.

ReagentX avatar ReagentX commented on August 17, 2024

Thanks again for the report, the above PR should address the issues.

from purple_air_api.

ReagentX avatar ReagentX commented on August 17, 2024

This release is now available in PyPI as version 1.1.

Note that the Blank field does appear to have some data:

>>> from purpleair.sensor import Sensor
>>> se = Sensor('2890')
>>> df = se.parent.get_historical(weeks_to_get=1,
...                               sensor_channel='b',
...                               thingspeak_field='primary')
>>> df['Blank'].value_counts()
100.0    3166
98.0      162
99.0       99
97.0       87
96.0       66
95.0       65
49.0       61
61.0       60
46.0       54
45.0       54
60.0       51
47.0       45
48.0       39
56.0       35
59.0       34
55.0       32
92.0       31
62.0       31
57.0       31
81.0       31
52.0       30
78.0       30
50.0       29
66.0       29
76.0       28
73.0       27
69.0       27
79.0       27
70.0       26
94.0       26
67.0       26
71.0       25
80.0       24
84.0       24
82.0       23
53.0       23
91.0       23
51.0       22
77.0       22
58.0       21
68.0       21
64.0       21
72.0       20
54.0       19
90.0       19
88.0       19
65.0       18
93.0       18
74.0       18
75.0       18
87.0       18
83.0       17
63.0       17
89.0       16
85.0       14
86.0       13
44.0        2
Name: Blank, dtype: int64

If there are any new problems please open a new issue.

from purple_air_api.

ReagentX avatar ReagentX commented on August 17, 2024

I was wrong about the above, the field is truly blank:

                        created_at  PM1.0_CF_ATM_ug/m3  PM2.5_CF_ATM_ug/m3  PM10.0_CF_ATM_ug/m3  UptimeMinutes  RSSI_dbm  Pressure_hpa  Blank  PM2.5_CF_1_ug/m3
entry_id                                                                                                                                                       
1002561  2020-09-09 00:01:09+00:00                1.03                1.41                 1.41        18136.0      0.01        982.25    NaN              1.41
1002562  2020-09-09 00:03:09+00:00                1.07                1.60                 1.60        18136.0      0.01        982.18    NaN              1.60
1002563  2020-09-09 00:05:09+00:00                1.28                1.59                 1.76        18136.0      0.01        982.10    NaN              1.59
1002564  2020-09-09 00:07:09+00:00                1.33                1.71                 1.71        18136.0      0.01        982.21    NaN              1.71
1002565  2020-09-09 00:09:09+00:00                1.25                1.86                 1.86        18136.0      0.01        982.18    NaN              1.86

This bug will be fixed in the 1.1.1 release.

from purple_air_api.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.