Git Product home page Git Product logo

sync's Introduction

Sync module for emoncms

Download and upload feeds between a local and remote emoncms server

syncmodule.png

  1. Login to your local installation of emoncms.
  2. Navigate to Setup > Sync
  3. Enter your emoncms.org username and password to bring up the list of available feeds.
  4. Click Download all, or download feeds individually
  5. Browse the feeds locally, that's it!

You may also be interested in the python based emoncms backup utility if you do not wish to setup a local emoncms instance to download/backup/archive emoncms data from a remote server such as emoncms.org: https://github.com/emoncms/usefulscripts/tree/master/backup_py

Automatic EmonPi, Emonbase Update

The Sync module is included in the default EmonPi/EmonBase software stack as of 26th April 2018. If you do not see the sync module under the emoncms Setup tab try running EmonPi or EmonBase update from the Administration page on your EmonPi/EmonBase.

Manual Linux Installation

Note: The sync module has hard coded paths for the emoncms directory location that point to: /var/www/emoncms. If your installation has emoncms installed in /var/www/html/emoncms you will need a symlink to /var/www/emoncms:

ln -s /var/www/html/emoncms /var/www/emoncms

The setting $home_dir or $emoncms_dir in emoncms settings.php also need to be set to reflect your system.

Install the sync module to /opt/emoncms/modules (you may need to create those directories first):

cd /opt/emoncms/modules
git clone https://github.com/emoncms/sync.git

Symlink the web part of the sync module to the public html emoncms modules directory:

ln -s /opt/emoncms/modules/sync/sync-module /var/www/emoncms/Modules/sync

Update the emoncms mysql database via the Admin > Update interface. Click on 'Update Database' next to the 'Update Database Only' option.

Install Service Runner

The sync module downloads or uploads data using a script that runs in the background. This script can be automatically called using the emoncms service runner. See Emoncms: Service-runner installation details here.

Troubleshooting

  • Check that the $homedir or $emoncms_dir setting is set appropriately on your emoncms installation
  • If your emoncms installation is in /var/www/html/emoncms, make a symlink to /var/www/emoncms as described above.

To run the sync process manually:

cd /opt/emoncms/modules/sync
sudo php sync_run.php

sync's People

Contributors

alexandrecuer avatar davefiddes avatar emrysr avatar glynhudson avatar reedy avatar thib66 avatar trystanlea avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

sync's Issues

Description issue

In development: Download or upload emoncms feed data between local an remote emoncms server's

an should be and

Only works with emoncms not self hosted server

Might be work making it clearer this will only work at the moment on the cloud emoncms.org not on a self hosted server to pi install.

are there any plans to make it work with the normal emoncms install to normal emoncms install?

Errors in Apache Logs

The following errors were seen in the apache logs when sync is selected (no config)

[Sat Feb 29 17:25:10.077296 2020] [php7:notice] [pid 2457] [client 192.168.7.123:65002] PHP Notice:  Trying to get property 'host' of non-object in /opt/emoncms/modules/sync/sync-module/sync_controller.php on line 51, referer: http://192.168.7.47/sync/view
[Sat Feb 29 17:25:10.077462 2020] [php7:notice] [pid 2457] [client 192.168.7.123:65002] PHP Notice:  Trying to get property 'apikey_read' of non-object in /opt/emoncms/modules/sync/sync-module/sync_controller.php on line 51, referer: http://192.168.7.47/sync/view
[Sat Feb 29 17:25:10.077518 2020] [php7:warn] [pid 2457] [client 192.168.7.123:65002] PHP Warning:  file_get_contents(/feed/listwithmeta.json?apikey=): failed to open stream: No such file or directory in /opt/emoncms/modules/sync/sync-module/sync_controller.php on line 51, referer: http://192.168.7.47/sync/view
[Sat Feb 29 17:25:10.077703 2020] [php7:warn] [pid 2457] [client 192.168.7.123:65002] PHP Warning:  Invalid argument supplied for foreach() in /opt/emoncms/modules/sync/sync-module/sync_controller.php on line 83, referer: http://192.168.7.47/sync/view

sync not working with emoncms v10

image
connexion seems to be OK. sync_model.php is connecting to the remote emoncms server (an old 9.8.28 version) and all necessary params (host, username, API read key, API write key) saved into the sync table of the mariadb server
Anyway, the sync screen remains blank


With a 9.9.9 version, things are fine
image
image
image
@glynhudson : Can you reproduce such a bug on your side ?

Feed names with whitespace, do not get uploaded to remote.

Our setup is running from an emoncms (running on Pi - using pre-built SD card), configured to sync to a remote, self-hosted online emoncms.

We have managed to connect the two services together - and we can see the local and remote feeds in the list. Feeds can be uploaded to the remote, online emoncms and we see data.

If a feed name contains any white-space; the feed fails to upload. The "Local ahead of Remote by xxx" counter slowly increments.

sync is not working with emoncms version 11.0.2

downloading a PHPFINA feed :

http://192.168.2.3/sync/download?name=temp&tag=TRH12211196_Cellule2&remoteid=56&interval=300&engine=5&datatype=1

return is :

Fatal error: Uncaught Error: Object of class stdClass could not be converted to string in /var/www/emoncms/Modules/feed/feed_model.php:126 Stack trace: #0 /var/www/emoncms/Modules/feed/feed_model.php(126): mysqli_stmt->execute() #1 /opt/emoncms/modules/sync/sync-module/sync_controller.php(152): Feed->create() #2 /var/www/emoncms/core.php(75): sync_controller() #3 /var/www/emoncms/index.php(242): controller() #4 {main} thrown in /var/www/emoncms/Modules/feed/feed_model.php on line 126

working fine with version 10.8.5 (stable)

return is success :

{"success":true}

missing object properties in feed-list

Attempted to sync my dev laptop to my locally running emonpi.
Inputs synced ok, however the feeds section didn't show the available feeds.

The api call /sync/feed-list returned this malformed json response:

Notice: Undefined property: stdClass::$start_time in [emoncms_dir]/sync/sync-module/sync_controller.php on line 67

[screenshot of empty feeds list]
delwedd

Unable to list feeds - alerts with "Error - 200: OK" with php 8.2

Discovered on a fresh install of Emoncms using emonScripts updated to support php8.2 on Debian 12 "bookworm". Navigating to the Sync page displays a JavaScript alert "Error - 200: OK" and does not list feeds when the connection details for another emoncms are provided.

Investigating the response for the /sync/feed-list request shows a php warning:

<br />
<b>Deprecated</b>:  Creation of dynamic property Sync::$feed is deprecated in <b>/opt/emoncms/modules/sync/sync-module/sync_model.php</b> on line <b>29</b><br />
{"enphase_panel\/Back_East_LRW":...}

Credentials not URL encoded

The remote username and password are not sanitised or encoded for posting via URL. I think this is causing the issue I am experiencing, which is that authentication fails against my remote server. My password is full of special characters that need encoded for POSTing.

According to the PHP documentation for curl_setopt's CURLOPT_POSTFIELDS setting, this post data must be URL encoded.

In general there are other places that should use encoding, e.g. here. Basically it should be used everywhere regardless of whether the data is expected to be safe or not.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.