residuum / purestjson Goto Github PK
View Code? Open in Web Editor NEWPuREST JSON is a library for connecting Puredata (Pd) to HTTP services and encoding and decoding JSON data.
Home Page: ix.residuum.org/pd/purest_json.html
License: Other
PuREST JSON is a library for connecting Puredata (Pd) to HTTP services and encoding and decoding JSON data.
Home Page: ix.residuum.org/pd/purest_json.html
License: Other
Although the Windows binaries are compiled with SSL support for libcurl, issuing SSL secured requests lead to segfaults for [rest] and [oauth].
Master won't compile because
json-encode.c
line 36 says:
#include "uthash/src/uthash.h"
but this file is not included in the github repository.
Google has changed their API, so the results are not very illuminating. There's lots of fake api sites for testing that might be useful instead:
http://jsonplaceholder.typicode.com/
is one such thing.
since you talked about doing a new release i thought it time to report this minor issue:
whenever an object from the library is instantiated, a nice boilerplate is displayed in the Pd-console:
[] part of PuREST JSON version
For more info see http://ix.residuum.org/pd/purest_json.html
now i'm a big fan of showing boilerplates and use them extensively in my own libraries.
however, i also see the concerns of those who feel like being drowned in unsolicited noise.
therefore, I'd like to suggest to not display the boilerplate whenever a new object ins instantiated, but instead reduce the number of shows to a single one.
this can most easily be achieved by moving the call to purest_json_lib_info()
from the object constructor (e.g. rest_new()
) to the setup function (e.g. rest_setup()
).
an alternative is to use a static int firsttime=1;
flag, but i don't see much benefits to do that.
[rest] and [oauth] should get a [header( method to set and unset HTTP headers.
just a general code hygiene advise to avoid symbol clashes:
it would be good to declare all functions that are not explicitely required to be accessible from outside as static
.
usually this means all functions except for the setup function (e.g. oauth_setup
) of an object (which of course must be visible for Pd to call into).
i see that you also setup APIEXPORT
to achieve something similar, although I don't understand why you want to export a function like e.g. json_decode_list
at all (it seems nowhere to be used except for single file src/json-decode.c).
apart from that: if you already have an APIEXPORT
you might want to use that on un*x-like systems as well and implement -fvisiblity=hidden
Replace these functions with calls to libcurl. As there will be a lot of shared code between [oauth] and [rest], readd a shared library.
Line 150 in ctw.c produces a crash on OSX 10.8:
pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, PTHREAD_CANCEL_ENABLE);
Commenting that line seems to work for simple tests ...
In the downloads section, when hitting on the windows download you get a file not found error.
Can you re-upload it please?
-bundle and -dynamiclib can't coexist in Mac OS X. It might also break stuff on GNU/Linux. When building on MinGW, just add the needed flags to SHARED_LDFLAGS rather than including ALL_LDFLAGS.
[oauth] only does OAuth 1.1. As version 2.0 of the standard is available, and is used by several large service providers (Facebook, Google+), [oauth] should either support version 2.0 or a different object is needed.
currently the path to the (included) uthash.h
header is hardcoded in ./src/json-encode.c
resp. ./src/urlparams.c
:
#include "uthash/src/uthash.h"
this makes it hard to use an uthash implementation that lives in a different place. (e.g. when building from the tarballs (which lack the git submodule) and using e.g. the Debian-provided package uthash-dev
)
a more generic approach would be to use:
#include "uthash.h"
in the source-code and add the path to the build-flags (in Makefile
):
ALL_CFLAGS = -I"$(PD_INCLUDE)" -Iuthash/src -std=c99
[rest] and [oauth] do work on Windows with the version that is compiled with MXE.
Error message:
... couldn't create
load_object: Symbol "rest_setup" not found
or
load_object: Symbol "oauth_setup" not found
Both functions are in the source files and in the header file, setup function for the other objects are found.
i stumbled across the "same" thing as #51 with the Debian package (which i maintain ๐คฆโโ๏ธ ) of PuRestJson.
Here the issue seems to be that json-decode
and json-encode
use is_error
, which is an undefined symbol with json-c >=0.13 (actually it's a define
in json-c/bits.h
, but this file is never included by PuRestJson)
all the other objects from this library seem to work fine.
I've created a new ticket even though this might be a dupe of #51, just in case (my problem is on Debian; and Debian only switched to json-c_0.13 in mid-2018)
How can I save encoded json data to external file?
thanks
husk
i'm currently uploaded a 'pd-purest-json' package to Debian.
as explained in Debian's Upstream Guide, I would like to ask you to remove the debian/
packaging directory from the repository, as it will become obsolete once the package is in Debian.
(you are of course welcome to only remove it, once the package has entered Debian; e.g. before your next release).
Hi. We are using PuRESTJson for the Freesound API:
https://github.com/g-roma/freesound_pd
Unfortunately we still need to resort to [shell] + curl for downloading files, which means it won't work on windows. Handling binary downloads (e.g. to file) in PuRESTJson would be a great improvement.
[oauth] and [json] are only working in 64bit pure data. Would be nice to be able to load them on a 32bit puredata instance as well so they can be usable along with many old libraries pd (gem etc...)
OAuth is not implemented yet. First of all, a clear strategy is needed.
From @megrimm via email:
When example [GET /1.1/users/search.json?q=puredata{ is sent to [oauth] to get data from Twitter API, not all data is returned therefore giving a "Not a JSON object" using [json-decode]. Not sure the issue if its [oauth] or me...
here the pd console output:
data1: [\{"id":12096842,"id_str":"12096842","name":"IBM Netezza ","screen_name":"IBMNetezza","location":"Marlborough, MA","description":"IBM PureData for Analytics dramatically simplifies high-performance data analytics across the enterprise.","url":"http:\\/\\/t.co\\/xKEICQwg1L","entities":\{"url":\{"urls":[\{"url":"http:\\/\\/t.co\\/xKEICQwg1L","expanded_url":"http:\\/\\/www.ibmbigdatahub.com","display_url":"ibmbigdatahub.com","indices":[0,22]\}]\},"description":\{"urls":[]\}\},"protected":false,"followers_count":6643,"friends_count":3324,"listed_count":359,"created_at":"Fri Jan 11 03:04:23 +0000 2008","favourites_count":4,"utc_offset":-28800,"time_zone":"Pacific Time (US & Canada)","geo_enabled":true,"verified":false,"statuses_count":4651,"lang":"en","status":\{"created_at":"Tue Jun 25 16:52:33 +0000 2013","id":349570821603340288,"id_str":"349570821603340288","text":"@crystaanderson @TheSocialPitt - c'mon, I said \\"Go Blackhawks\\" surely that's enough #bruins","source":"\\u003ca
Not a JSON object
status1: bang
Convert json objects to the new json-c API, and keep the older version, because it is still widely used.
When querying some webservices, the [rest-json]
is not able to parse the JSON data, e.g. take the Twitter examples.
[rest] currently implements GET, POST, PUT, DELETE.
The remaining HTTP verbs PATCH, HEAD, OPTIONS, CONNECT, TRACE should be implemented in later versions.
s
MinGW does not come with pkg-config. So, all dependencies must be listed explicitely for Windows. Alternatively, list steps to install pkg-config on Windows.
Hello. First at all, this external is very, very nice, so much thanks for sharing it!
I 'm using as a Twitter client, pretty much like the example and it works flawlessly, although when i 'm behind a proxy, It does not work. I suppose could be because pd-extended cannot see the windows or O.S proxy config ? is there a way to declar the proxy config/path in the patch ?
many thanks in advance
I've tried this both on Mac and a Raspberry Pi 4 after installing from "Find Externals"
Thanks!
I open rest-json-help.pd, replace [print data] with [json-decode] and click one of the URLs, and it crashes every time for me. I'm using Pd-extended 0.43.1 20120119 64-bit build on Mac OS X 10.6.8. purest_json is built against json-c 0.9 and libcurl 7.24.0.
0 libjson.0.dylib 0x0000000106a95bdc json_object_put + 12
1 pd 0x0000000100027782 outlet_list + 82
2 libpurest_json.dylib 0x0000000106a9fac9 output_json + 473 (libpurest_json.c:48)
3 rest-json.pd_darwin 0x0000000106aa374c execute_rest_thread + 284 (rest-json.c:163)
4 libSystem.B.dylib 0x000000010017dfd6 _pthread_start + 331
5 libSystem.B.dylib 0x000000010017de89 thread_start + 13
Using the 32-bit Mac OS X build with , I also get a similar crash:
0 libjson.0.dylib 0x190e609a json_object_put + 10
1 pd 0x0002ccad outlet_list + 77
2 libpurest_json.dylib 0x190eedd9 output_json + 665
3 ??? 0x0604d200 0 + 100979200
4 ??? 0x65736e6f 0 + 1702063727
When a HTTP request issued by [rest] or [oauth ]does not return anything, and not [timeout( is set, then the object remains locked forever.
And maybe someone wants to be able to cancel a request, because something has changed, and e.g. the data that should be stored in a CouchDB instance needs to be updated before storing it.
[rest] and [oauth] should both implement a [cancel( message to abort a pthread that is currently running and unlock the object.
The object "rest" does not support streaming IO. It does not send output when the server does not close the connection.
For example GET requests on http://stream-sandbox.oanda.com/v1/prices?instruments=EUR_USD do not work. Unfortunately I cannot attach a pd-file. If you have any problems reproducing the behaviour, I can attach a jpg.
When overwriting existing data in [json-encode], adding an value to an array does not clear the previously stored data.
The output should be:
print: symbol { "id": 1.000000 }
print: symbol { "id": [ 1.000000, 2.000000 ] }
print: symbol { "id": 2.000000 }
But instead is:
print: symbol { "id": 1.000000 }
print: symbol { "id": [ 1.000000, 2.000000 ] }
print: symbol { "id": [ 1.000000, 2.000000 ] }
RSA signing of OAUth requests is not implemented.
Working on an ARM7 computer (NTC CHIP) running a debian 4.3 kernel.
I opened this file with json_encode and tried to decode it. PD 046-7 crashes. Yes, it's quite big, as far as JSON files go. Using purest json installed with apt-get from stretch repo.
Arrays of dictionaries don't seem to be handled properly, e.g.:
{
'a':1,
'b':[
{'c':2,'d':3},
{'e':3}]
}
After decoding, the following sequence:
[json-decode]
|
[list trim]
|
[route b]
|
[json-decode]
|
[print result]
Will print a flat list, i.e c,d and e at the same level.
@megrimm reported via Email:
hmmmm...
as soon as I try this on a production machine (not dev machine i compiled on) I get error in the pd console with "the-sound-of-money.pd" example:
Error while performing request: Problem with the SSL CA cert (path? access rights?)
....what could this be? libcurl.dylib permission stuff? im not getting too much info from a quick google search....
osx 10.9
m
The example with cookie authentication does not work correctly, the header data is not written correctly.
liboauth not being linked correctly osx 10.9. the following error occurs in pd console
oauth
... couldn't create
/Users/megrimm/Library/Pd/purest_json/oauth.pd_darwin: dlopen(/Users/megrimm/Library/Pd/purest_json/oauth.pd_darwin, 10): Symbol not found: _oauth_sign_url2
Referenced from: /Users/megrimm/Library/Pd/purest_json/oauth.pd_darwin
Expected in: dynamic lookup
When loading purest_json in MinGW I get the error "pd.exe - system error": "The program can not be started, because libpurest_json.dll is missing on the computer."
I have got this idea via email.
There is https://libwebsockets.org/, which is LGPL. It is available in Debian (and Ubuntu), OS X has it with homebrew and MXE for Windows cross-compilation has it as well.
No object in PuREST JSON implements WebDAV.
The following verbs are needed in addition to existing ones:
PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, LOCK, UNLOCK
The deken package for 1.4.1 is missing cacert.pem.
if you use the message [add key#$ value(, then the output should be key%23%24=value.
Similar to issue #27, but all remaining HTTP verbs should work with [oauth] as well.
[oauth] currently implements GET, POST.
The remaining HTTP verbs PUT, DELETE, PATCH, HEAD, OPTIONS, CONNECT, TRACE should be implemented in later versions.
Hi everyone, I'm having this issue:
I built the library as explained in your instructions.
rest object works fine, but PD cannot create json-encode and json-decode objects.
I'm running pd 0.48-1 on macOS 10.13.4
Any hint?
Thank you
running the build multiple times, results in an error:
$ make clean
++++ info: using Makefile.pdlibbuilder version 0.4.4
++++ info: making target clean in lib purest_json
rm -f src/json-decode.o src/json-encode.o src/oauth.o src/rest.o src/urlparams.o src/purest_json.o
rm -f json-encode.pd_linux rest.pd_linux urlparams.pd_linux oauth.pd_linux json-decode.pd_linux purest_json.pd_linux
rm -f *.pre *.lst
$ make
++++ info: using Makefile.pdlibbuilder version 0.4.4
++++ info: using Pd API /usr/include/pd/m_pd.h
++++ info: making target all in lib purest_json
++++ info: making src/json-encode.o in lib purest_json
[...]
++++ info: linking objects in json-decode.pd_linux for lib purest_json
cc -rdynamic -shared -fPIC -Wl,-rpath,"\$ORIGIN",--enable-new-dtags -o json-decode.pd_linux src/json-decode.o -lc -lm -lcurl -ljson-c -loauth
++++info: target all in lib purest_json completed
$ make
++++ info: using Makefile.pdlibbuilder version 0.4.4
++++ info: using Pd API /usr/include/pd/m_pd.h
++++ info: making target all in lib purest_json
++++ info: evaluating implicit prerequisites in lib purest_json.....
cc: error: src/json-decode: Datei oder Verzeichnis nicht gefunden
cc: fatal error: no input files
compilation terminated.
pd-lib-builder/Makefile.pdlibbuilder:921: *** missing separator. Stop.
@katjavetter this is probably an issue with pd-lib-builder, rather than PuRestJson.
i've also tested with pd-lib-builder 0.5.0 (using #45), with the same result:
$ make PDLIBBUILDER_DIR=/usr/share/pd-lib-builder
++++ info: using Makefile.pdlibbuilder version 0.5.0
++++ info: using Pd API /usr/include/pd/m_pd.h
++++ info: making target all in lib purest_json
++++ info: evaluating implicit prerequisites in lib purest_json.....
cc: error: src/json-decode: No such file or directory
cc: fatal error: no input files
compilation terminated.
/usr/share/pd-lib-builder/Makefile.pdlibbuilder:959: *** missing separator. Stop.
I will split the [rest-json] object:
When adding a value, all numbers are converted to double. Integers should be added as integers, so that [add id 1( results in {"id": 1} as string, but not {"id": 1.000} as now.
When posting data with [oauth], no data is transmitted.
"purest_json: can't load library"
Raspberry Pi3, Raspbian Stretch, deken purest-json-v1.4.1, compiled pd 0.48.1
Should I compile the library against my pd 0.48.1? How to do it?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.