Git Product home page Git Product logo

Bigfile ———— a file transfer system that supports http, rpc and ftp protocol 简体中文English


Bigfile is a file transfer system, supports http, ftp and rpc protocol. Designed to provide a file management service and give developers more help. At the bottom, bigfile splits the file into small pieces of 1MB, the same slice will only be stored once.

In fact, we built a file organization system based on the database. Here you can find familiar files and folders.

Since the rpc and http protocols are supported, those languages supported by grpc and other languages can be quickly accessed.

More detailed documents can be found here

Features

  • Support HTTP(s) protocol

    • Support rate limit by ip
    • Support cors
    • Support to avoid replay attack
    • Support to validate parameter signature
    • Support Http Single Range Protocol
  • Support FTP(s) protocol

  • Support RPC protocol

  • Support to deploy by docker

  • Provide document with English and Chinese

Install Bigfile

There are kinds of ways to install Bigfile, you can find more detailed documentation here English 简体中文

Start Bigfile

  1. generate certificates

bigfile rpc:make-cert

  1. start server

bigfile multi:server

This will print some information as follows:

    [2019/09/19 15:38:32.817] 56628 DEBUG  bigfile http service listening on: https://0.0.0.0:10985
    [2019/09/19 15:38:32.818] 56628 DEBUG   Go FTP Server listening on 2121
    [2019/09/19 15:38:32.819] 56628 DEBUG  bigfile rpc service listening on: tcp://[::]:10986

HTTP Example (Token Create)

package main

import (
	"fmt"
	"io/ioutil"
	libHttp "net/http"
	"strings"
	"time"

	"github.com/bigfile/bigfile/databases/models"
	"github.com/bigfile/bigfile/http"
)

func main() {
	appUid := "42c4fcc1a620c9e97188f50b6f2ab199"
	appSecret := "f8f2ae1fe4f70b788254dcc991a35558"
	body := http.GetParamsSignBody(map[string]interface{}{
		"appUid":         appUid,
		"nonce":          models.RandomWithMD5(128),
		"path":           "/images/png",
		"expiredAt":      time.Now().AddDate(0, 0, 2).Unix(),
		"secret":         models.RandomWithMD5(44),
		"availableTimes": -1,
		"readOnly":       false,
	}, appSecret)
	request, err := libHttp.NewRequest(
		"POST", "https://127.0.0.1:10985/api/bigfile/token/create", strings.NewReader(body))
	if err != nil {
		fmt.Println(err)
		return
	}
	request.Header.Set("Content-Type", "application/x-www-form-urlencoded")
	resp, err := libHttp.DefaultClient.Do(request)
	if err != nil {
		fmt.Println(err)
		return
	}
	if bodyBytes, err := ioutil.ReadAll(resp.Body); err != nil {
		fmt.Println(err)
		return
	} else {
		fmt.Println(string(bodyBytes))
	}
}

RPC Example (Token Create)

package main

import (
	"crypto/tls"
	"crypto/x509"
	"fmt"
	"io/ioutil"

	"google.golang.org/grpc"
	"github.com/bigfile/bigfile/rpc"
	"google.golang.org/grpc/credentials"
)

func createConnection() (*grpc.ClientConn, error) {
	var (
		err           error
		conn          *grpc.ClientConn
		cert          tls.Certificate
		certPool      *x509.CertPool
		rootCertBytes []byte
	)
	if cert, err = tls.LoadX509KeyPair("client.pem", "client.key"); err != nil {
		return nil, err
	}

	certPool = x509.NewCertPool()
	if rootCertBytes, err = ioutil.ReadFile("ca.pem"); err != nil {
		return nil, err
	}

	if !certPool.AppendCertsFromPEM(rootCertBytes) {
		return nil, err
	}

	if conn, err = grpc.Dial("192.168.0.103:10986", grpc.WithTransportCredentials(credentials.NewTLS(&tls.Config{
		Certificates: []tls.Certificate{cert},
		RootCAs:      certPool,
	}))); err != nil {
		return nil, err
	}
	return conn, err
}

func main() {
	var (
		err  error
		conn *grpc.ClientConn
	)

	if conn, err = createConnection(); err != nil {
		fmt.Println(err)
		return
	}
	defer conn.Close()
	grpcClient := rpc.NewTokenCreateClient(conn)
    	fmt.Println(grpcClient.TokenCreate(context.TODO(), &rpc.TokenCreateRequest{
    		AppUid:    "42c4fcc1a620c9e97188f50b6f2ab199",
    		AppSecret: "f8f2ae1fe4f70b788254dcc991a35558",
    }))
}

FTP Example

ftp-example

FOSSA Status Get it from the Snap Store

bigfile's Projects

bigfile icon bigfile

Bigfile -- a file transfer system that supports http, rpc and ftp protocol https://bigfile.site

ftp-server icon ftp-server

ftp-server clones from https://gitea.com/goftp/server

homebrew-tap icon homebrew-tap

Homebrew Formulae to @bigfile binaries, powered by @bigfile

scoop-bucket icon scoop-bucket

Scoop bucket for @bigfile binaries, powered by @bigfile

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.