Git Product home page Git Product logo

sockethub / sockethub Goto Github PK

View Code? Open in Web Editor NEW
377.0 25.0 46.0 12.29 MB

A multi-protocol gateway for the Web using ActivityStream messages.

Home Page: http://sockethub.org

License: GNU Lesser General Public License v3.0

JavaScript 26.27% Dockerfile 0.14% TypeScript 64.30% Lua 2.08% CSS 0.24% HTML 0.09% Svelte 6.88%
activity-stream unhosted xmpp irc redis websockets socket-io sockethub javascript nodejs

sockethub's Introduction

Sockethub

Sockethub

A protocol gateway for the web.

Compliance CodeQL Maintainability Release

About

Sockethub is a translation layer for web applications to communicate with other protocols and services that are traditionally either inaccessible or impractical to use from in-browser JavaScript.

Using ActivityStream (AS) objects to pass messages to and from the web app, Sockethub acts as a smart proxy server/agent, which can maintain state, and connect to sockets, endpoints, and networks that would otherwise, be restricted from an application running in the browser.

Originally inspired as a sister project to RemoteStorage, and assisting in the development of unhosted and noBackend applications, Sockethub's functionality can also fit into a more traditional development stack, removing the need for custom code to handle various protocol specifics at the application layer.

Example uses of Sockethub are:

  • Writing and receiving messages (SMTP, IMAP, Nostr ...)

  • Chat (XMPP, IRC, SimpleX, ...)

  • Discovery (WebFinger, RDF(a), Link preview generation ...)

The architecture of Sockethub is extensible and supports easy implementation of additional 'platforms' to carry out tasks.

Docs

See the Sockethub wiki for documentation.

Features

We use ActivityStreams to map the various actions of a platform to a set of AS '@type's which identify the underlying action. For example, using the XMPP platform, a friend request/accept cycle would use the activity stream types 'request-friend', 'remove-friend', 'make-friend'.

Below is a list of platform contexts we're currently working on and their types, both the completed and not yet implemented ones. They are all implemented in Sockethub platforms (each in their own repository) and can be enabled/disabled in the config.json.

Platforms

Making a platform is as simple as creating a platform module that defines a schema and a series of functions that map to verbs. Take a look at some of our existing platforms for examples.

Run

To get up and running quickly, you only need the following commands:

pnpm install
pnpm dev

Dependencies

pnpm install

Build

pnpm build

Tests

pnpm test

Linter

pnpm lint:js

Or, to automatically fix linting errors:

pnpm lint:fix

Integration Tests

pnpm integration

Start

For development purposes, with examples enabled, run:

DEBUG=sockethub* pnpm dev

You should then be able to browse to http://localhost:10550 and try out the examples.

For production, with examples disabled.

DEBUG=sockethub* pnpm start

For more info on configuration options, see the Sockethub README section on environment variables.*

Packages

Credits

Project created and maintained by Nick Jennings

Logo design by Jan-Christoph Borchardt

Sponsored by NLNET

NLNET Logo

sockethub's People

Contributors

abliss avatar bitdeli-chef avatar bkero avatar cobertos avatar dependabot[bot] avatar galfert avatar ggrin avatar gregkare avatar michielbdejong avatar mscdex avatar nilclass avatar ramsingla avatar raucao avatar renovate-bot avatar renovate[bot] avatar silverbucket avatar timoschilling avatar tosbackcrawler avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sockethub's Issues

XMPP: properly log out of xmpp

it keeps saying

[DEBUG - 2013-05-11T14:45:19.016Z]  [listener:xmpp] waiting for cleanup to finish...
[DEBUG - 2013-05-11T14:45:20.019Z]  [listener:xmpp] waiting for cleanup to finish...
[DEBUG - 2013-05-11T14:45:21.019Z]  [listener:xmpp] waiting for cleanup to finish...

and hangs. then when i Ctrl-c it, it does:

[DEBUG - 2013-05-11T14:45:24.023Z]  [listener:xmpp] waiting for cleanup to finish...
[WARN  - 2013-05-11T14:45:24.359Z]  [dispatcher] platform xmpp hasnt responded to ping
[ERROR - 2013-05-11T14:45:24.359Z]  [dispatcher] listeners failed to respond to ping, aborting.

use of follow verb

Wondering if you have any feedback on this @michielbdejong or @nilclass

I'd like to implement 'follow', however I'm caught in a bit of a paradox about how it should be used.

  • On one hand, it has been suggested it's used to indicate subscribing to something: follow RSS feed, follow a Twitter user
  • On the other hand, it's also indicative of receiving the actual updates themself: follow your email account to get new messages.

How are these technically the same things? If we issue a follow command to an RSS feed, does this mean there has to be constant polling of the RSS feed to send new posts to the user right away? What about when the user signs out of sockethub? Are we then 'unfollowing' the RSS feed? This doesn't make a huge amount of sense as you should only follow the RSS feed once, right? Then you need a new verb to describe retrieving the messages.

Perhaps there's a need for a poll command which is different than the follow command? Polling your RSS feeds would give you all updates. But then what does subscribing (following) offer? And how does this behave across sessions?

For email, does it make sense, then, to 'follow' your email? Or do you just poll?

For twitter or facebook. Again, whats the point of 'follow' when, in the end, you'll need to poll for updates to your feed. And how do those follow's persist across sessions?

If you've given this any thought I'd be curious to know what your take on this is. I have trouble correllating these activity streams paradigms to actual implementations sometimes.

"cleanup" function for platforms

When I implement a platform, I may need to keep some kind of state between messages (such as persistent connections in some cases - e.g. xmpp).
All this needs to be cleaned up once the websocket connection is closed.

So to make this work I propose that platforms can expose an optional "cleanup" function, which is called once a connection is closed.

Example:

var stuff = {};

exports.cleanup = function(sessionId) {
  if(stuff[sessionId]) {
    // do some cleaning up if needed, then
    delete stuff[sessionId];
  }
}

evaluate pump.io

i don't see mailing list so put it here for now :)

http://pump.io
(by Evan Prodromou who previously worked on StatusNet)

"What's it for?

I post something and my followers see it. That's the rough idea behind the pump.

There's an API defined in the API.md file. It uses activitystrea.ms JSON as the main data and command format.

You can post almost anything that can be represented with activity streams -- short or long text, bookmarks, images, video, audio, events, geo checkins. You can follow friends, create lists of people, and so on.

The software is useful for at least these scenarios:

Mobile-first social networking
Activity stream functionality for an existing app
Experimenting with social software

"

server stops if you send it an unrecognized command

/root/run/sockethub/lib/protocols/sockethub/dispatcher.js:373
var report = jsv.validate(o, proto.verbs[verb].schema);
^
TypeError: Cannot read property 'schema' of undefined
at WebSocketConnection.Dispatcher.connect.connection.sendUTF.JSON.stringify.rid (/root/run/sockethub/lib/protocols/sockethub/dispatcher.js:373:57)
at WebSocketConnection.EventEmitter.emit (events.js:93:17)
at WebSocketConnection.processFrame (/root/run/sockethub/node_modules/websocket/lib/WebSocketConnection.js:403:26)
at WebSocketConnection.handleSocketData (/root/run/sockethub/node_modules/websocket/lib/WebSocketConnection.js:247:14)
at Socket.EventEmitter.emit (events.js:93:17)
at TCP.onread (net.js:396:14)

it should catch this and reply 'command unrecognized', 'platform unrecognized', 'unparseable', etcetera. nothing you throw at it should cause it to topple.

how can i use the secret?

i am trying to use sockethub to execute shell commands as per https://unhosted.org/adventures/6/Controlling-your-server-over-a-WebSocket.html . can you add a minimal example to html/index.html that shows how to execute 'whoami' or 'ls' or whatever when the user clicks a button? I have this working in https://github.com/michielbdejong/sockethub, but i used a secret in each command instead of in register, and i would like to switch to your way of doing it, but i can't fully see how it works from the current code.

Cheers!
Michiel.

EMAIL: pop3 support

i'm gonna go with https://github.com/ditesh/node-poplib for POP3. i can't think of a good ActivityStreams verb, so i'll just go with 'retrieve' for now. i think all platforms can deliver activities they retrieve to the 'inbox' module on remoteStorage, if the sockethub server has access to that.

we can switch this sort of things around as we see how it looks, obviously, but i want to have something working for my blogpost tomorrow.

directory structure of sockethub repo

Most of the code is nested deeply in lib/protocols/sockethub/(platforms/). I would suggest to instead have something like:

lib/dispatcher.js
lib/listener.js
lib/session.js
lib/server/http.js // was lib/httpServer
lib/server/ws.js
lib/schema/protocol.js
lib/schema/verbs.js
lib/platforms/...

Then possibly protocol.js could be renamed platforms.js, because it primarily aggregates information about platforms from lib/platforms/ (also most of the info in there could be gathered automatically, by loading the platforms and examining their exports).

What do you think?

tls support for outgoing email

i ran into this because i'm in a hotel that blocks outgoing port 25, so sockethub times out when trying to send an email.

they don't block port 465 though, meaning that sending over tls should be possible. but afaik there is no way to specify that in the actor credentials?

registering takes like 20 seconds

i'm continuously restarting sockethub with code changes, and then my all needs to connect (on the websocket level) and register again, obviously. not sure why, but this is taking a very long time. how is this for you?

allow for posting to targets on facebook

right now i'm doing something like

https://github.com/sockethub/sockethub/pull/39/files

when posting to https://www.facebook.com/unhostedwebapps

otherwise you can only post on your own wall, which for the way people use facebook in practice is really not enough imho.

as you can see in the closed pull request, i don't have working code for this with tests and schemas, sorry.

add a way to retrieve contacts

i know the syntax for this is still up in the air, but i went ahead and created a 'fetch' verb that allows you to retrieve the list of facebook friends of the current user:
michielbdejong@919d2fd

i need this for this week's Unhosted Adventures episode, which will be about contacts.

email account not showing up when i set its credentials

when i set the credentials config in the facebook tab of html/index.html, a 'facebook account' appears. i would expect the same thing on the email tab, but it somehow doesn't, i keep having a dropdown with zero accounts in it on the email tab

no response given when uninitiated platform is called

sockethub should do a better job of letting the client know that a command failed because the platform was not initialized, rather than silently dropping the job like it does now (added to a queue that is never read from).

put some content on sockethub.org and announce the project on the ML

i would like to link from https://unhosted.org/ to http://sockethub.org/ and also use that URL instead of http://github.com/sockethub/sockethub when mentioning the project. Could you maybe put at least the logo and a link to the repo on there? also, i mentioned your project in last week's Adventures episode, but i guess it would be useful to start a thread about it on https://groups.google.com/forum/#!forum/unhosted maybe? Just an idea :)

Cheers!
Michiel.

support for drop-in platforms

it would be nice if i could distribute a platform, like the webrtc.js i'm working on right now, and distribute it on its own as a sort of plugin/module for sockethub. just drop the file into lib/protocols/sockethub/platforms/, and maybe activate it in config.js, and it could work without further code changes.

now i have to go through the 'burocracy' as i call it, editing protocol.js to manually add the platform and the verbs it support.

i understand you need this when introducing new verbs, because of the schemas and everything, but if a platform only implements existing verbs, then it would be nice if you could just drop it in without editing the core code.

i thought that was also the plan originally, that it would work that way?

Dispatcher.errorSent lasts whole session

if an error was sent once, then nothing will work anymore until you close the WebSocket and reconnect it.

i think errorSent should be reset to false at the start of on('message')?

Proper handling of platform specific credentials

cc @nilclass @michielbdejong

Currently there's not defined way of passing credentials to a platform, and keeping them active for the session. Niklas has a working XMPP proof of concept, and in this initial implementation it seems he just passes the credentials along with 'message' verb. (BTW - if this is what we decide to use it should be specified in the schema, and examples).

The bonus to this, is that we can easily use different accounts without needing to establish some sort of 'sub-session' within a sockethub session, and the platform does not need to maintain state of any kind. It just handles each request from the beginning to end.

The downside, is also that there's no real state for the platform. So you wouldn't be able to do any polling for updates to send to the client (unless the platform kept active using the credentials from a 'message' command, but this kind of implicit behavior can be misleading).

We could have some sort of 'init' verb, which establishes the identity of the 'actor'. The identity could be encapsulated in the username, and stored in a platform specific internal object. This way, sending new messages, you would still specify the 'actor' and based on that address, the platform could look up existing credential details or a previously received token, and use that to post messages. The user could switch between different accounts, simply by specifying the different 'actor', as long as that actor was previously 'init'ialized.

Is init a good name for it? authenticate? authorize? register (might be a bad idea to use this with a different meaning that the initial sockethub register).

What do you guys think? Feedback much appreciated.

Use checkin to set/receive status of actors for the xmpp platform

Once you've set the credentials for your XMPP platform, you must 'checkin' to log in. If you are not logged in, but try to send a message, it will also automatically log you in, but it's better to log in first with this command:

{
  "platform": "xmpp",
  "verb": "checkin",
  "actor": {
    "address": "[email protected]",
  },
  "object": {
    "state": "online",    // online, offline, dnd, away
    "statusText": null,
  }
}

tests for xmpp platform

hey @nilclass any chance I could get you to write some tests for xmpp support? there's a teste tool "Mock" which would allow you to write some mock function for any xmpp stuff that requires external server/connections.

error trace from node_mailer

when trying to send an email (using the email app), i get:

node_modules/mailer/lib/node_mailer.js:59
      if(client == hostClients[options.user]) {
                   ^
ReferenceError: hostClients is not defined
    at SMTPClientPool.send (/media/mich/remotestorage/owncloud-freedombox/admin
/open_web_apps/code/projects/sockethub/node_modules/mailer
/lib/node_mailer.js:59:20)
    at SMTPClient.EventEmitter.emit (events.js:93:17)
    at SMTPClient._createConnection (/media/mich/remotestorage/owncloud-
freedombox/admin/open_web_apps/code/projects/sockethub/node_modules/mailer
/node_modules/nodemailer/lib/smtp.js:669:14)
    at Socket.EventEmitter.emit (events.js:96:17)
    at Socket._destroy.destroyed (net.js:358:10)
    at process.startup.processNextTick.process._tickCallback (node.js:244:9)

not sure what's going wrong here, is sending email working for you? also, it kills sockethub.js which shouldn't happen, it should just stay up and report back 'error while executing command' or something

add pgp support

i'm using sockethub to send out pgp-signed email, here is my code:

https://github.com/sockethub/sockethub/pull/40/files#L1R22

sorry that i didn't have time to do the whole pull-request thing with the tests and the schemas and the docs and everything, i have lots of other stuff piling up this week, but i wanted to at least share my code with you, you're free to use it however works best.

hopefully it will at least serve as example code for how pgp signing can work. this is the code i use in production for actual real email sending. :)

fwiw i plan to spend some time helping with actual sockethub development in May, until then i'll just be your user base and send you support tickets! :)

simplify target schema

for email, we have an overly complicated target schema:

"target" : [
  "to": [
    {
      "address": "[email protected]",
    }
  ],
  "cc": [
    {
      "address": "[email protected]"
    }
  ],
  "bcc": []
]

In other cases, however, this doesn't make any sense. Instead let's simplify this like so:

"target" : [  
  {
    "field": "to",
    "address": "[email protected]",
  },
  {
    "field": "cc",
    "address": "[email protected]"
  }
]

Suggestions for something other than 'field' would be welcome :) @nilclass @michielbdejong

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.