Git Product home page Git Product logo

spee.ch's Introduction

Spee.ch

spee.ch provides a user-friendly, custom-designed, image and video hosting site backed by a decentralized network and blockchain (LBRY). Via just a small set of config files, you can spin your an entire spee.ch site back up including assets.

Please note: the spee.ch code base and setup instructions are no longer actively maintained now that we have lbry.tv. Proceed at your own caution. Setup will require dev ops skills.

App GIF

For a completely open, unrestricted example of a spee.ch site, check out https://www.spee.ch.

For a closed, custom-hosted and branded example, check out https://lbry.theantimedia.com/.

Installation

Ubuntu Step-by-Step

Step-by-step Ubuntu Install Guide

Full Instructions

Get some information ready:

Install and Set Up Dependencies

  • Firewall open ports
    • 22
    • 80
    • 443
    • 3333
    • 4444
  • NodeJS
  • MySQL version 5.7 or higher
    • mysqlusername or root
    • mysqlpassword
    • Requires mysql_native_password plugin
    mysql> `ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'yourpassword';`
    
  • lbrynet daemon
    • run this as a service exposing ports 3333 and 4444
    • note: once the daemon is running, issue commands in another terminal session (tmux) to retrieve an address for your wallet to recieve 5+ LBC credits (or join us in the #speech discord channel and we will send you a few)
      • ./lbrynet commands gets a list of commands
      • ./lbrynet account_balance gets your balance (initially 0.0)
      • ./lbrynet address_list gets addresses you can use to recieve LBC
  • FFmpeg
  • ImageMagick
  • Spee.ch (below)
  • pm2 (optional) process manager such as pm2 to run speech server.js
  • http proxy server e.g. caddy, nginx, or traefik, to forward 80/443 to speech port 3000
    • note: even running on http://localhost, you must redirect http or https to port 3000

Clone spee.ch

  • release version for stable production
$ git clone -b release https://github.com/lbryio/spee.ch.git
  • master version for development
$ git clone https://github.com/lbryio/spee.ch.git
  • your own fork for customization

Change directory into your project

$ cd spee.ch

Install node dependencies

$ npm install

Create the config files using the built-in CLI

Make sure lbrynet is running in the background before proceeding.

note: If you are opt to run a local chainquery, such as from lbry-docker/chainquery you will need to specify connection details at this time in: ~/spee.ch/docs/setup/conf/speech/chainqueryConfig.json

$ npm run configure

Build & start the app

$ npm run build

$ npm run start

View in browser

Customize your app

Check out the customization guide to change your app's appearance and components

(optional) add custom components and update the styles

  • Create custom components by creating React components in site/custom/src/
  • Update or override the CSS by changing the files in site/custom/scss

(optional) install your own chainquery

Instructions are coming at [lbry-docker] to install your own chainquery instance using docker-compose. This will require 50GB of preferably SSD space and at least 10 minutes to download, possibly much longer.

Settings

There are a number of settings available for customizing the behavior of your installation.
Here is some documentation on them.

API

/api/claim/publish

method: POST

example:

curl -F 'name=MyPictureName' -F 'file=@/path/to/myPicture.jpeg' https://spee.ch/api/claim/publish

Parameters:

  • name (required, must be unique across the instance)
  • file (required) (must be type .mp4, .jpeg, .jpg, .gif, or .png)
  • nsfw (optional)
  • license (optional)
  • title (optional)
  • description (optional)
  • thumbnail URL to thumbnail image, for .mp4 uploads only (optional)
  • channelName channel to publish too (optional)
  • channelPassword password for channel to publish too (optional, but required if channelName is provided)

response:

{
    "success": <bool>,
    "message": <string>,
    "data": {
        "name": <string>,
        "claimId": <string>,
        "url": <string>,
        "showUrl": <string>,
        "serveUrl": <string>,
        "lbryTx": {
            "claim_address": <string>,
            "claim_id": <string>,
            "fee": <number>,
            "nout": <number>,
            "tx": <string>,
            "value": <number>
        }
    }
}

/api/claim/availability/:name

method: GET

example:

curl https://spee.ch/api/claim/availability/doitlive

response:

{
    "success": <bool>,  // `true` if spee.ch successfully checked the claim availability
    "data": <bool>, // `true` if claim is available, false if it is not available
    "message": <string> // human readable message of whether claim was available or not
}

Contribute

Stack

The spee.ch stack is MySQL, Express.js, Node.js, and React.js. Spee.ch also runs lbrynet on its server, and it uses the lbrynet API to make requests -- such as publish, create_channel, and get -- on the LBRY network.

Spee.ch also runs a sync tool, which decodes blocks from the LBRY blockchain as they are mined, and stores the information in MySQL. It stores all claims in the Claims table, and all channel claims in the Certificates table.

Architecture

  • cli/ contains the code for the CLI tool. Running the tool will create .json config files and place them in the site/config/ folder

    • configure.js is the entry point for the CLI tool
    • cli/defaults/ holds default config files
    • cli/questions/ holds the questions that the CLI tool asks to build the config files
  • client/ contains all of the client code

    • The client side of spee.ch uses React and Redux
    • client/src/index.js is the entry point for the client side js. It checks for preloaded state, creates the store, and places the <App /> component in the document.
    • client/src/app.js holds the <App /> component, which contains the routes for react-router-dom
    • client/src/ contains all of the JSX code for the app. When the app is built, the content of this folder is transpiled into the client/build/ folder.
      • The Redux code is broken up into actions/ reducers/ and selectors/
      • The React components are broken up into containers/ (components that pull props directly from the Redux store), components/ ('dumb' components), and pages/
      • spee.ch also uses sagas which are in the sagas/ folders and channels/
    • client/scss/ contains the CSS for the project *
  • site/custom is a folder which can be used to override the default components in client/

    • The folder structure mimics that of the client/ folder
    • to customize spee.ch, place your own components and scss in the site/custom/src/ and site/custom/scss folders.
  • server/ contains all of the server code

    • index.js is the entry point for the server. It creates the express app, requires the routes, syncs the database, and starts the server listening on the PORT designated in the config files.
    • server/routes/ contains all of the routes for the express app
    • server/controllers/ contains all of the controllers for all of the routes
    • server/models/ contains all of the models which the app uses to interact with the MySQL database.
      • Spee.ch uses the sequelize ORM for communicating with the database.
  • tests/ holds the end-to-end tests for this project

    • Spee.ch uses mocha with the chai assertion library
    • unit tests are located inside the project in-line with the files being tested and are designated with a xxxx.test.js file name

Tests

  • This package uses mocha with chai for testing.
  • Before running tests, create a testingConfig.js file in devConfig/ by copying testingConfig.example.js
  • To run tests:
    • To run all tests, including those that require LBC (like publishing), simply run npm test
    • To run only tests that do not require LBC, run npm run test:no-lbc

URL formats

Spee.ch has a few types of URL formats that return different assets from the LBRY network. Below is a list of all possible URLs for the content on spee.ch. You can learn more about LBRY URLs here.

Dependencies

Spee.ch depends on two other lbry technologies:

  • chainquery - a normalized database of the blockchain data. We've provided credentials to use a public chainquery service. You can also install it on your own server to avoid being affected by the commons.
  • lbrynet - a daemon that handles your wallet and transactions.

Bugs

If you find a bug or experience a problem, please report your issue here on GitHub and find us in the lbry discord!

License

This project is MIT licensed. For the full license, see LICENSE.

Security

We take security seriously. Please contact [email protected] regarding any security issues. Our GPG key is here if you need it.

Contact

The primary contact for this project is @jessopb.

spee.ch's People

Contributors

alessandrospallina avatar alyssaoc avatar bones7242 avatar caseyparker avatar d1v3 avatar daovist avatar dependabot[bot] avatar dhruvamsharma avatar filipnyquist avatar jessopb avatar joshstrobl avatar kauffj avatar kcseb avatar lavradis avatar lbrydocs avatar lyoshenka avatar maximest-pierre avatar minesh93 avatar msftserver avatar nathan78906 avatar nikooo777 avatar piste avatar professordey avatar quirkyrobots avatar rizdaprasetya avatar skhameneh avatar stoatally avatar strikerrus avatar tzarebczan avatar ykris45 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

spee.ch's Issues

serve multiple image sizes

If client indicates the size of the image requested, provide the asset in that size.

Examples:

  • if client requests spee.ch/example/h40, spee.ch serves an image with height of 40 (while maintaining aspect ratio)
  • if client requests spee.ch/example/w60, spee.ch serves an image with width of 60 (while maintaining aspect ratio)
  • if client requests spee.ch/example/40x60, spee.ch serves an image with height of 40 and width of 60 (stretching original image)

add show routes

User story: As a user, I want to be able to view images in my browser along with metadata and tools, so that I can learn more about the image and easily share/embed it where I need to.

Add a 'show' route for claims. This route will display the image in a web page as opposed to serving the asset directly. The show route page will include a couple simple tools (copy to clipboard, etc).

After publish, spee.ch should redirect the user to the show route.

deal with timeouts on serve requests

story: as a user, I want to know quickly whether the requested content is unavailable, so that I do not have to wait and then be dissapointed.

Some get requests to daemon are resulting in timeouts which the daemon is getting confused by.

When the daemon times out, it will hang and then send a timeout message. (note: shortening the timeout by specifying it in the get parameters does not work atm. I specified 30 seconds but it still takes 180 to timeout, which seems to be what creates the 'hanging'). Once the daemon has fully timed out, it sends the timeout error.

When the request is made a second time, the daemon will log that it is already downloading the file (2017-06-30 15:35:03,701 INFO lbrynet.daemon.Daemon:1513: Already waiting on lbry://example to start downloading) and then responds to the client with the following: "{\"id\":null,\"jsonrpc\":\"2.0\",\"result\":null}"

This seems to be an error with the daemon, but spee.ch should be able to handle this by detecting the timeout and then responding to the client with further direction. Unfortunately, the subsequent calls that return the result of null are a little more tricky to deal with.

speechbot

user story: As a slack user, I want to be able to type !speech lake and get the claim from spee.ch for the name 'lake' (https://spee.ch/lake), so that I don't have to type out the full url.

A bot to provide interaction with spee.ch from slack. The ideal bot will be great for lbry, but also for other slack communities who can install it and benefit from the fun/helpful tools it provides.

@filipnyquist is working on this.

Publishing does not outbid existing claims

Currently when you publish a new image with a name matching an existing claim, it does not outbid that claim and the publish process continues as though it has worked correctly.

It should either:

  1. Outbid any existing claims (who pays?)
  2. Complain to the uploader that the claim is taken.

I'm curious if named claims are really the best way forward, it sure is cool to be able to name your image, but is it practical to implement it that way, and does it solve any real world problem?

Simplify homepage

Strip out the non-essential info from the homepage and move it to a subpage.

update 'claim name' input to be more intuitive

Story: As a user, I want to know that the claim name is the url at which my image will be published, so that I know what the input is for.

solution: prefix the input box with 'spee.ch/'

rename files on the server side

Right now spee.ch is stopping the user from submitting files with certain file names. Spee.ch should allow all file names through the upload portion and then rename those files on the server side instead.

back-end name, uri, and filetype validation

The back end should validate the url params received by the routes before initiating action on them. The daemon will reject them, but it would be better to catch it up front.

For example: checking for correct characters in the name, checking the length of the uri, etc.

The affected routes include:

  • api routes (also needs to validate file type)
  • serve routes
  • applicable 'show' routes

investigate why spee.ch links don't unfurl on twitter

story: as a user, when I share a spee.ch link on twitter I want twitter to load the image from the link so that my followers can see the image without having to click through the link.

issue: At the moment, twitter does not retrieve the asset and display it in your tweet, it just shows the link.

steps to reproduce (from @finer9 ) in twitter

  1. click image icon in new tweet box
  2. in file chooser dialog box paste https://spee.ch/url.jpg here
  3. It fetches the location and uploads it almost like a local image

for sharing links, add buttons with options for autoplay, looping, etc. and have the embedded

story: As a user who publishes a video, I want to be able to change the options on the video and have the html embed code change for me so I can adapt my video for my desired use without knowing html.

for sharing links, add buttons with options for autoplay, looping, etc. and have the html snippet adapt accordingly.

options to offer

  • loop on/off
  • autoplay on/off
  • width size: 100% or ___ px
  • iframe vs. html video tag

Acceptance Criteria

Definition of Done

  • Tested against acceptance criteria
  • Tested against the assumptions of the user story
  • The project builds without errors
  • Unit tests are written and passing
  • Tests on devices/browsers listed in the issue have passed
  • QA performed & issues resolved
  • Refactoring completed
  • Any configuration or build changes documented
  • Documentation updated
  • Peer Code Review performed

implement continuous integration between this repo and spee.ch server.

Story: As a developer, I want to be able to commit my changes to master and have those changes go live, so that I do not have to spend time manually updating the version on the server.

Acceptance Criteria

Definition of Done

  • Tested against acceptance criteria
  • Tested against the assumptions of the user story
  • The project builds without errors
  • Unit tests are written and passing
  • Tests on devices/browsers listed in the issue have passed
  • QA performed & issues resolved
  • Refactoring completed
  • Any configuration or build changes documented
  • Documentation updated
  • Peer Code Review performed

add link for support and feedback

as a user I want to know where to submit feedback and ask for support

  • feedback should be directed to spee.ch slack or an email address
  • support requests should be directed to an email address or github issues
  • feature requests should be directed to github issues

add 'markdown' as a share option

Story: As a user, I want to be able to copy a markdown text version of my url when I publish my image, so I can share my image.

Example below

markdown example

meme generator

add a meme generator, based off of the publish tool, as its own page.

create plan for stress testing server

Story: As developers, we want to test the capabilities of the server in responding to requests, so that we can ensure our user's requests will be handled without failure.

  • set up a method of documenting server performance metrics
  • find a resource to simulate stress test and compare metrics
  • document the stress-test process for future tests

accurate audience locations in analytics

Story: as a developer, I want to see how many visitors are coming from each country, so that I can better understand spee.ch's audience.

Issue: google analytics were showing all 'serve' requests coming from Virginia.

Solution: I believe this is happening because google is receiving some of spee.ch's server's data rather than the user's. I will adjust the universal-analytics params to ensure it is sending data about the request and not spee.ch.

Front end input validaton

Story: As a user, I want to be told when my input is not acceptable in a way that does not disrupt my user experience, so that I can easily complete my goal of submitting the input.

Front end validation on inputs (specifically the 'name' field) is letting in "_" which should not be allowed. Regex needs to be updated.
Validation is also using alerts to tell users of invalid characters. This needs to be changed to something user friendly, like red text above the offending inputs.

Publish Error: Rejected by Network Rules (18: bad-txns-inputs-spent)

user story: As a user, I want to be able to publish images from the spee.ch homepage and through spee.ch's api, so that I can share those images across the internet.

issue: On publish, users are receiving Network Rules (18: bad-txns-inputs-spent) error.

This appears to be an issue with the wallet and daemon, and perhaps with a race condition in which transactions are not being recorded correctly. It has been reported on lbry here lbryio/lbry-sdk#737.

A short term fix that has worked so far is removing the wallet and creating a new one, but that only seems to solve the issue for a short while.

It is unknown whether this is result purely from the volume of transactions or as a result of setting the claim_address in the publish params (which was a fix to the issue where every publish created a new wallet-address).

Images that have been updated do not refresh on spee.ch

Certain images that spee.ch serves have been updated elsewhere via lbry but the updated file does not show up on spee.ch.

This appears to be a result of spee.ch's daemon thinking it already has the most recent image and not retrieving the updated one. As seen below, a resolve call to the spee.ch daemon will show the updated outpoint. However, a get call to the same uri will return the old asset with the old outpoint. The daemon logs (see below) confirm that it believes it already has the most recent file and I believe is therefore not going out and re-downloading the file from lbry.

I have not yet reported an issue on lbryio/lbry. @kaykurokawa mentioned this may have to do with caching.

$ curl 'http://localhost:5279/lbryapi' --data '{"method":
"resolve","params":{"uri":"meme-fodder#61bcc109366c2b66c1507866211fa8d7ea53bb9b"} }'

{
  "id": null,
  "jsonrpc": "2.0",
  "result": {
    "meme-fodder#61bcc109366c2b66c1507866211fa8d7ea53bb9b": {
      "claim": {
        "address": "bJpWLpQwq5xdQ6EzW2TPUkvf3B8v3MQxvt",
        "amount": 2000000,
        "claim_id": "61bcc109366c2b66c1507866211fa8d7ea53bb9b",
        "claim_sequence": 2,
        "decoded_claim": true,
        "depth": 282,
        "effective_amount": 1000000,
        "has_signature": false,
        "height": 199308,
        "hex": "080110011a9b0108011254080410011a0b6d656d652d666f6464657222216d65
6d652d666f64646572207075626c69736865642076696120737065652e63682a07737065652e6368
320d5075626c696320446f6d61696e38004a0052005a001a41080110011a3098e731bf406b0440a5
0f3d1bdc29036139c8b76db61f9cce127e471cacde21af7526000206143b80465d3971080e134522
09696d6167652f706e67",
        "name": "meme-fodder",
        "nout": 0,
        "supports": [],
        "txid": "6aeb24bde5389064e036888b9efdad5b87bcadf49228e83f99e5ac3ca9b4406
0",
        "valid_at_height": 199314,
        "value": {
          "claimType": "streamType",
          "stream": {
            "metadata": {
              "author": "spee.ch",
              "description": "meme-fodder published via spee.ch",
              "language": "en",
              "license": "Public Domain",
              "licenseUrl": "",
              "nsfw": false,
              "preview": "",
              "thumbnail": "",
              "title": "meme-fodder",
              "version": "_0_1_0"
            },
            "source": {
              "contentType": "image/png",
              "source": "98e731bf406b0440a50f3d1bdc29036139c8b76db61f9cce127e471
cacde21af7526000206143b80465d3971080e1345",
              "sourceType": "lbry_sd_hash",
              "version": "_0_0_1"
            },
            "version": "_0_0_1"
          },
          "version": "_0_0_1"
        }
      }
    }
  }
}



$ curl 'http://localhost:5279/lbryapi' --data '{"method":
"get","params":{"uri":"meme-fodder#61bcc109366c2b66c1507866211fa8d7ea53bb9b"} }
'

{
  "id": null,
  "jsonrpc": "2.0",
  "result": {
    "claim_id": "61bcc109366c2b66c1507866211fa8d7ea53bb9b",
    "completed": true,
    "download_directory": "/home/lbry/Downloads",
    "download_path": "/home/lbry/Downloads/owl.jpeg",
    "file_name": "owl.jpeg",
    "has_signature": false,
    "key": "9b4f23be86dbc5568e9f997caeebf05c",
    "message": "Started owl.jpeg, got 1/1 blobs, stream status: completed",
    "metadata": {
      "claimType": "streamType",
      "stream": {
        "metadata": {
          "author": "spee.ch",
          "description": "meme-fodder published via spee.ch",
          "language": "en",
          "license": "Public Domain",
          "licenseUrl": "",
          "nsfw": false,
          "preview": "",
          "thumbnail": "",
          "title": "meme-fodder",
          "version": "_0_1_0"
        },
        "source": {
          "contentType": "image/png",
          "source": "98e731bf406b0440a50f3d1bdc29036139c8b76db61f9cce127e471cacd
e21af7526000206143b80465d3971080e1345",
          "sourceType": "lbry_sd_hash",
          "version": "_0_0_1"
        },
        "version": "_0_0_1"
      },
      "version": "_0_0_1"
    },
    "mime_type": "image/jpeg",
    "name": "meme-fodder",
    "outpoint": "4849f7e567641ef23db735ae9ad485b660e41619d7e58beac92c5739f8ebdb9
8:0",
    "points_paid": 0.0,
    "sd_hash": "0c38f31b8050a10d438fba060e6e622beca6cebd5cc8d50c29eb2597a4097312
effec6f3a658b254b429b0509cda72bf",
    "stopped": true,
    "stream_hash": "c6c1627aecf755c2e4a2d4b0e390d9f868f31e7ec4dcc007383b0811e3ea
c33b5da3fe62b432fdae452a12e6499d38c4",
    "stream_name": "owl.jpeg",
    "suggested_file_name": "owl.jpeg",
    "total_bytes": 28032,
    "written_bytes": 28026
  }
}

(from the Daemon logs: )
Jun 30 18:44:39 ip-172-55-1-123 lbrynet-daemon[911]: 2017-06-30 18:44:39,761 INFO lbrynet.core.Wallet:810: Resolving meme-fodder#61bcc109366c2b66c1507866211fa8d7ea53bb9b
Jun 30 18:44:39 ip-172-55-1-123 lbrynet-daemon[911]: 2017-06-30 18:44:39,896 INFO lbrynet.daemon.Daemon:1524: Already have a file for meme-fodder

add image extensions to url

As a user, I want to get a speech url with my image's extension on it so I can easily tell that it is a direct link to that image.

names that yeild no free, public claims should not be treated as errors

Currently spee.ch's logic returns an error to the controller if a free, public claim is not found, and then the router serves a page with a 307 status code telling client there are no claims. This is not technically an error, it is a successful request that returned no results. The functionality should stay the same, but the architecture should be adjusted so this is not treated as an error.

Balanced display of trending images

Story: As a user, I want the trending images to be displayed in a nice orderly fashion of somewhat uniform size so that they are pleasant to the eye.

publish sharing links

User story: As a user, after I publish an image, I want to be shown the direct link to that image and be able to copy it easily, so that I can share or embed it elsewhere on the internet.

After a publish is complete, it should display links/buttons for sharing the published asset.

Currently the copy-to-clipboard button is broken. That needs to be fixed.

It should also offer 'tweet', 'post to reddit', embedding options (like an image tag using your spee.ch url), and other helpful sharing buttons.

Maximum file size

At the moment it's possible to publish a file of any size, and also download a file of any size. We should probably add a sensible limit to both, something around 5 to 10MB maybe?

Store publishes in user session

As a spee.ch user, sometimes I upload my image and then accidentally close the tab or forget what URL I used. It would be useful if I could see my recent publishes (or really all of my publishes, as long as my cookie hasn't changed).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.