Git Product home page Git Product logo

device's Introduction

Emoncms

Emoncms is an open-source web application for processing, logging and visualising energy, temperature and other environmental data and is part of the OpenEnergyMonitor project.

Emoncms

Requirements

  • PHP (tested with 8.1.12)
  • MySQL or MariaDB (tested with 10.5.15)
  • Apache (tested with 2.4.54)
  • Redis* (tested with 6.0.16)

*Redis is recommended because it reduces the number of disk writes and therefore prolongs disk life (noticeably on SD cards e.g. Raspberry Pi). Some input-processors also require Redis and fail silently if Redis is not installed. Some environments such as shared hosting or as far as we have tried Windows servers don't support Redis hence why Emoncms has a fall back mode that allows core operation without Redis.

Documentation

View the Emoncms documentation at: https://docs.openenergymonitor.org/emoncms

Design

Emoncms timeseries database design (feed storage)

Other

Emoncms Terminology

  • Input: An incoming datasource. Each input has an associated "node" identifier and a "key" sub-identifier. Inputs are entry points, only the last value and time of the input is recorded. To record historic data a feed needs to be created from an input.
  • Input: Node: A grouping identifier for an input or feed.
  • Input: Key: A sub-identifier for items within each Node.
  • Input process list (or input processing): A list of processes* performed sequentially on each input value as it is received on that input.
  • Process: A function that can be attached to the process list of an input to change the value or to save the value to a feed*.
  • Feed: A place where data is recorded, a time-series of datapoints. The standard time-series databases used by Emoncms are PHPFina and PHPTimeSeries and were written as part of the Emoncms project.
  • For a description of what each input process does in Emoncms, see the helper note within the Emoncms input processing configuration interface.

Emoncms.org API Reference

Install

Emoncms is designed and tested to run on either Ubuntu Linux (Local, Dedicated machine or VPS) or RaspberryPi OS. It should work on other Debian Linux systems though we dont test or provide documentation for installation on these.

We do not recommend and are unable to support installation on shared hosting or XAMPP servers, shared hosting in particular has no or limited capabilities for running some of the scripts used by emoncms. There is now a large choice of low cost miniature Linux VPS hosting solutions that provide a much better installation environment at similar cost.

Recommended:

docker standalone container

An easy way to start with emoncms is to use the all-in-one docker container

A pipeline using github actions is producing builds with latest emoncms stable version for different architectures : amd64, arm64, arm/v7

These docker images, based on the alpine linux distribution, are designed for iot. Images are tagged using alpine and emoncms versions, for example alpine3.19_emoncms11.4.11.

The images have onboard :

  • the mariadb and redis databases,
  • the mosquitto mqtt broker,
  • the main modules : graph, sync, backup, dashboard and app,
  • the workers : emoncms_mqtt, service-runner and feedwriter.

You can easily :

  • deactivate the low-write
  • use an external broker.

To pull the latest image for testing :

sudo docker pull alexjunk/emoncms

More on https://emoncms-docker.github.io

Experimental

not currently up to date

Multi-platform using Docker Container

Modules

Modules can be installed by downloading or git cloning into the emoncms/Modules folder. Be sure to check for database updates in Administration menu after installing new modules. The following core modules are included on the emonSD image:

There are many other available modules such as the event module and openbem (open source building energy modelling module): check out the Emoncms repo list.

Branches

  • master - The latest and greatest developments. Potential bugs, use at your own risk! All pull-requests should be made to the master branch.

  • stable - emonPi/emonBase release branch, regularly merged from master. Slightly more tried and tested. See release change log.

Tools

  • PHPFina data file viewer - Easily explore phpfina timeseries feed engine data files directly without a full Emoncms installation. Useful for checking backups and archived data.

Android App

Google Play

GitHub Repo

Development Forum

More information

device's People

Contributors

adminde avatar chaveiro avatar circuitsetup avatar emrysr avatar glynhudson avatar myforest avatar openastroproject avatar pb66 avatar reedy avatar thib66 avatar tomtom5152 avatar trystanlea avatar veggie2u avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

device's Issues

Processes only accessible via the depreciated mechanism

To make the templates easier to read I have tried using the function name in place of the processlist index number eg "process":"log_to_feed" rather than "process":"1" but that doesn't seem to work.

I have also tried "process":"process__log_to_feed" to try and match the "process":"additionalp__andfunction" which does work in the events module, so it appears to be that the original processes are only available via the numeric index that has apparently only been kept for backwards compatability, is that correct?

if so how difficult would it be to make them available via the function names?

Reinitializing device duplicates inputs

I had some devices posting to emoncms before making up the device template files, since devices are now created for all new inputs, there were new devices already listed on the device page so rather than deleting, I "edited" the device(s) and selected my new template and "reinitialized" the device.

The first time I did this it duplicated the inputs and I put that down to it not recognizing each existing input by it's index+1 temporary name given to it.

However when I fixed a typo to a feed name and added another input to the template and then reinitialized the same device a second time, it created a third set of inputs, see below.

image

It should have only created a single addition input as input "8" aka "frequency" was omitted in the draft template.

This creation of a third set of inputs is definitely a bug, but the second set although it could be the same error, I sort of expected it to happen as it couldn't map the old temp names to the new names, but now we have indexedinputs there should be no problem with mapping.

Confusing line in Readme

After updating a device template redis redis must be updated reload the templates to make the changes visable:

redis seems to be duplicated, and potentially missing some punctuation?

After updating a device template, redis must be updated. Reload the templates to make the changes visable:

Unable to add new templates (again)

This has been discovered, debugged and fixed previously on the forum (https://community.openenergymonitor.org/t/development-devices-inputs-and-feeds-in-emoncms/4281?u=pb66).

I am currently running emoncms 9.8.31 indexedinputs branch fully up to date at 15/10/18 with the "device-integration beta" switched off. The device module is also fully up to date and I've tried both the master and the indexedinputs branches.

It is currently not possible to introduce any new templates. I have tried adding new templates to both existing and new groups and new folders and every combination of those fails. At each change I have cleared the browser cache.

I have also tried creating a unwanted device to try and pull in the changes (this was previously found to work before this same issue was fixed last time), but the listed device templates doesn't change.

The only way I'm able to get the new template picked up is to delete all keys from Redis via the command line, then the list repopulates with the new templates included.

Errors in emoncms.log

Errors in emoncms.log related to device module from Dave's test emonPi:

1 /var/www/emoncms/Modules/device/device_model.php(280): Device->init_template(7) #2 /var/www/emoncms/scripts/phpmqtt_input.php(257): Device->autocreate(1, 'smartplug', 'smartplug') #3 [internal function]: message(Object(Mosquitto\Message)) #4 /var/www/emoncms/scripts/phpmqtt_input.php(116): Mosquitto\Client->loop() #5 {main} 2018-01-24 11:53:00.826|ERROR|phpmqtt_input.php|exception 'ErrorException' with message 'Missing argument 2 for Device::init_template(), called in /var/www/emoncms/Modules/device/device_model.php on line 280 and defined' in /var/www/emoncms/Modules/device/device_model.php:543 Stack trace: #0 /var/www/emoncms/Modules/device/device_model.php(543): exceptions_error_handler(2, 'Missing argumen...', '/var/www/emoncm...', 543, Array) #1 /var/www/emoncms/Modules/device/device_model.php(280): Device->init_template(7) #2 /var/www/emoncms/scripts/phpmqtt_input.php(257): Device->autocreate(1, 'smartplug', 'smartplug') #3 [internal function]: message(Object(Mosquitto\Message)) #4 /var/www/emoncms/scripts/phpmqtt_input.php(116): Mosquitto\Client->loop() #5 {main} 2018-01-24 21:54:50.805|WARN|phpmqtt_input.php|Not connected, retrying connection 2018-01-24 21:54:50.831|WARN|phpmqtt_input.php|Connecting to MQTT server: Connection Accepted.: code: 0 2018-01-24 22:17:35.137|WARN|phpmqtt_input.php|Not connected, retrying connection 2018-01-24 22:17:35.191|WARN|phpmqtt_input.php|Connecting to MQTT server: Connection Accepted.: code: 0 2018-01-31 05:47:43.418|ERROR|phpmqtt_input.php|exception 'ErrorException' with message 'Missing argument 2 for Device::init_template(), called in /var/www/emoncms/Modules/device/device_model.php on line 267 and defined' in /var/www/emoncms/Modules/device/device_model.php:530 Stack trace: #0 /var/www/emoncms/Modules/device/device_model.php(530): exceptions_error_handler(2, 'Missing argumen...', '/var/www/emoncm...', 530, Array) #1 /var/www/emoncms/Modules/device/device_model.php(267): Device->init_template(7) #2 /var/www/emoncms/scripts/phpmqtt_input.php(257): Device->autocreate(1, 'smartplug', 'smartplug') #3 [internal function]: message(Object(Mosquitto\Message)) #4 /var/www/emoncms/scripts/phpmqtt_input.php(116): Mosquitto\Client->loop() #5 {main} 1 /var/www/emoncms/Modules/device/device_model.php(280): Device->init_template(7) #2 /var/www/emoncms/scripts/phpmqtt_input.php(257): Device->autocreate(1, 'smartplug', 'smartplug') #3 [internal function]: message(Object(Mosquitto\Message)) #4 /var/www/emoncms/scripts/phpmqtt_input.php(116): Mosquitto\Client->loop() #5 {main} 2018-01-24 11:53:00.826|ERROR|phpmqtt_input.php|exception 'ErrorException' with message 'Missing argument 2 for Device::init_template(), called in /var/www/emoncms/Modules/device/device_model.php on line 280 and defined' in /var/www/emoncms/Modules/device/device_model.php:543 Stack trace: #0 /var/www/emoncms/Modules/device/device_model.php(543): exceptions_error_handler(2, 'Missing argumen...', '/var/www/emoncm...', 543, Array) #1 /var/www/emoncms/Modules/device/device_model.php(280): Device->init_template(7) #2 /var/www/emoncms/scripts/phpmqtt_input.php(257): Device->autocreate(1, 'smartplug', 'smartplug') #3 [internal function]: message(Object(Mosquitto\Message)) #4 /var/www/emoncms/scripts/phpmqtt_input.php(116): Mosquitto\Client->loop() #5 {main} 2018-01-24 21:54:50.805|WARN|phpmqtt_input.php|Not connected, retrying connection 2018-01-24 21:54:50.831|WARN|phpmqtt_input.php|Connecting to MQTT server: Connection Accepted.: code: 0 2018-01-24 22:17:35.137|WARN|phpmqtt_input.php|Not connected, retrying connection 2018-01-24 22:17:35.191|WARN|phpmqtt_input.php|Connecting to MQTT server: Connection Accepted.: code: 0 2018-01-31 05:47:43.418|ERROR|phpmqtt_input.php|exception 'ErrorException' with message 'Missing argument 2 for Device::init_template(), called in /var/www/emoncms/Modules/device/device_model.php on line 267 and defined' in /var/www/emoncms/Modules/device/device_model.php:530 Stack trace: #0 /var/www/emoncms/Modules/device/device_model.php(530): exceptions_error_handler(2, 'Missing argumen...', '/var/www/emoncm...', 530, Array) #1 /var/www/emoncms/Modules/device/device_model.php(267): Device->init_template(7) #2 /var/www/emoncms/scripts/phpmqtt_input.php(257): Device->autocreate(1, 'smartplug', 'smartplug') #3 [internal function]: message(Object(Mosquitto\Message)) #4 /var/www/emoncms/scripts/phpmqtt_input.php(116): Mosquitto\Client->loop() #5 {main}
1 /var/www/emoncms/Modules/device/device_model.php(280): Device->init_template(7) #2 /var/www/emoncms/scripts/phpmqtt_input.php(257): Device->autocreate(1, 'smartplug', 'smartplug') #3 [internal function]: message(Object(Mosquitto\Message)) #4 /var/www/emoncms/scripts/phpmqtt_input.php(116): Mosquitto\Client->loop() #5 {main} 2018-01-24 11:53:00.826|ERROR|phpmqtt_input.php|exception 'ErrorException' with message 'Missing argument 2 for Device::init_template(), called in /var/www/emoncms/Modules/device/device_model.php on line 280 and defined' in /var/www/emoncms/Modules/device/device_model.php:543 Stack trace: #0 /var/www/emoncms/Modules/device/device_model.php(543): exceptions_error_handler(2, 'Missing argumen...', '/var/www/emoncm...', 543, Array) #1 /var/www/emoncms/Modules/device/device_model.php(280): Device->init_template(7) #2 /var/www/emoncms/scripts/phpmqtt_input.php(257): Device->autocreate(1, 'smartplug', 'smartplug') #3 [internal function]: message(Object(Mosquitto\Message)) #4 /var/www/emoncms/scripts/phpmqtt_input.php(116): Mosquitto\Client->loop() #5 {main} 2018-01-24 21:54:50.805|WARN|phpmqtt_input.php|Not connected, retrying connection 2018-01-24 21:54:50.831|WARN|phpmqtt_input.php|Connecting to MQTT server: Connection Accepted.: code: 0 2018-01-24 22:17:35.137|WARN|phpmqtt_input.php|Not connected, retrying connection 2018-01-24 22:17:35.191|WARN|phpmqtt_input.php|Connecting to MQTT server: Connection Accepted.: code: 0 2018-01-31 05:47:43.418|ERROR|phpmqtt_input.php|exception 'ErrorException' with message 'Missing argument 2 for Device::init_template(), called in /var/www/emoncms/Modules/device/device_model.php on line 267 and defined' in /var/www/emoncms/Modules/device/device_model.php:530 Stack trace: #0 /var/www/emoncms/Modules/device/device_model.php(530): exceptions_error_handler(2, 'Missing argumen...', '/var/www/emoncm...', 530, Array) #1 /var/www/emoncms/Modules/device/device_model.php(267): Device->init_template(7) #2 /var/www/emoncms/scripts/phpmqtt_input.php(257): Device->autocreate(1, 'smartplug', 'smartplug') #3 [internal function]: message(Object(Mosquitto\Message)) #4 /var/www/emoncms/scripts/phpmqtt_input.php(116): Mosquitto\Client->loop() #5 {main}

Prompt to user to select a device template is non is chosen

Currently, if a user doesn't select any device template which is easy to miss since there are no clear instructions or prompt to do so and the device template list is hidden on a mobile device, the user is still able to click save without selecting any template.

How about an error message if a user tried to click 'save' without selecting a template, or 'save' could be greyed out until a suitable template has been selected?

Question about device templates held in Redis

Why does every template seem to have 2 superfluous key:value pairs?

Whilst trying to navigate the Redis entries for device templates (related to #24) I noticed that every template has the following

1) "module"
2) "device"

. . . 

11) "control"
12) ""

fields 2 thru 10 seem valid enough, but these don't seem to serve an obvious purpose (that I can see)

pb66@test2:$ redis-cli HGETALL emoncms:device:template:emonpi-SPV1
 1) "module"
 2) "device"
 3) "name"
 4) "Solar PV Type 1"
 5) "category"
 6) "OpenEnergyMonitor"
 7) "group"
 8) "EmonPi"
 9) "description"
10) "EmonPi Solar PV Type 1 template"
11) "control"
12) ""

Generate DeviceKey gives error

When editing a device and clicking on New to generate a new device key it gives this error:


Fatal error: Uncaught Error: Call to a member function hSet() on boolean in /home/gtdev/sites/emondev.gtronica.com/Modules/device/device_model.php:563Stack trace:#0 /home/gtdev/sites/emondev.gtronica.com/Modules/device/device_controller.php(98): Device->set_new_devicekey(32)#1 /home/gtdev/sites/emondev.gtronica.com/core.php(63): device_controller()#2 /home/gtdev/sites/emondev.gtronica.com/index.php(240): controller('device')#3 {main} thrown in /home/gtdev/sites/emondev.gtronica.com/Modules/device/device_model.php on line 563

Change nodeid for inputs when renaming node

When re-naming a node (screenshot); the node's inputs are not adjusted to use the new node name. The old node is then re-created when the next input data is received.

delwedd

Node field above is the device.nodeid field in MySQL
Name field above is the device.name field in MySQL

MySQL data structure: device.nodeid == input.nodeid

Would be good to add the check for node inputs and editing their input.nodeid property in the following lines: https://github.com/emoncms/device/blob/master/device_model.php#L435

As inputs is a Module in https://github.com/emoncms/emoncms it would be better to use the input controller to achieve this without having to change the input Module directly.

Custom and pre-defined template sets

Could we consider having the data directory structure sub-divided rather than all the templates in one folder?

That way users/we can better manage custom sets of templates that meet our own needs and standard templates can be grouped and maintained in their own repo's, eg OEM stuff in one repo, the existing "amig.json" and "amrm.json" etc in another so that we can install the templates we want and more importantly, maintain our own repo of custom templates without them having to share a folder with the included templates.

A similar sort of set up to the emoncms Modules folder where a minimum of just an "example" folder is included in the emoncms/device repo and a .gitignore file to prevent installed template folders from being committed when contributing.

The folder names could also double up as grouping names so that as the number of templates increases they are still easily found.

This might encourage other device/software developers to maintain their own device templates to make using their products even easier to use with emoncms.

I would be happy to look at this if it's an acceptable idea, I'm not specifically asking for someone to do it, just asking if anyone has any opinion at this stage

Readme?

What is this module for? Could it have a readme?

I'm intrigued! .... :-)

Miss create (new) device button

When setup of a new device you will need a devicekey ahead of the first post.
The create new device button is missing from the UI.
It was available on the old version.

Type 1 vs Type 2 clarification on Configure Devices window

While the image at https://guide.openenergymonitor.org/images/applications/solar-pv/solar-pv-install.webp on https://guide.openenergymonitor.org/applications/solar-pv/ explains what Type 1 vs Type 2, it's not immediately obvious what those mean from the "Configure Device" window

Screenshot 2022-01-30 at 01 55 26

I've no idea if HTML link parsing will work from the JSON files (not tested yet)... But having some sort of reference link there to https://guide.openenergymonitor.org/applications/solar-pv/#sensor-installation would potentially be useful for those browsing around the interface who may wonder what Type 1 vs Type 2 are

Support an Input/Device without a device template.

Should be possible to configure the input: name, location and even devicekey without having a device template.

Currently it gives the error: "Device need to be configured first"

You may add an option like: "Configure Manually" when there is no template available so that processors and feeds need to be configured manually.

It's like the old way of emoncms working, inputs get created automatically when a device posts.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.