Git Product home page Git Product logo

ysb33r's Introduction

Groovy VFS

A DSL for Groovy to wrap around the Apache Commons VFS libraries.

If you like it, then tweet about it using #groovyvfs as the hashtag.

Build Status Windows Build Status

Groovy Library

Download', link=

@Grapes([
	@Grab( 'org.ysb33r.groovy:groovy-vfs:1.0.1' ),
	@Grab( 'commons-net:commons-net:3.+' ), // If you want to use ftp
    @Grab( 'commons-httpclient:commons-httpclient:3.1'), // If you want http/https
    @Grab( 'com.jcraft:jsch:0.1.48' ) // If you want sftp
])
import org.ysb33r.groovy.dsl.vfs.VFS

def vfs = new VFS()

// Simple copy operation
vfs.cp 'ftp://foo.example/myfile', 'sftp://bar.example/yourfile'

// Utilising the DSL
vfs {

    // Copy file from one site to anther using two different protocols
    cp 'http://first.example/myfile', 'sftp://second.example/yourfile'

    // Not implemented yet - move file between two sites using different protocols
    mv 'sftp://second.example/yourfile', 'ftp://third.example/theirfile'

    // Lists all files on a remote site
    ls ('http://first.example') {
        println it.name
    }

    // Streams the output
    cat ('http://first.example/myfile') { strm->
        println strm.text
    }

    // Create a new folder on a remote site
    mkdir 'sftp://second.example/my/new/folder'

    // Change default options via property Map
    options 'vfs.ftp.passiveMode' : true

    // Change default options DSL style
    options {
        ftp {
            passiveMode true
        }
    }

    // Use options on a per URL basis
    cp 'ftp://first.example/myfile?vfs.ftp.passiveMode=1', 'sftp://second.example/yourfile?vfs.sftp.compression=zlib'

    // Download a compressed archive and unpack to local directory
    cp 'tbz2:ftp:/first.example/myFiles.tar.bz2", new File( '../unpack-here' ), recursive:true

    // Replace content of a file with text
    overwrite 'ftp://first.example/myfile?vfs.ftp.passiveMode=1' with 'this text'
    overwrite 'ftp://first.example/myfile?vfs.ftp.passiveMode=1', { strm -> strm << 'this text' }

    // Append content to a file
    append 'ftp://first.example/myfile?vfs.ftp.passiveMode=1' with 'this text'
    append 'ftp://first.example/myfile?vfs.ftp.passiveMode=1', { strm -> strm << 'this text' }
}

Gradle plugin

Download

From initiation a VFS object has been available as an extension to the project class. The interface is very experimental and may change without much warning in future releases of this plugin.

buildscript {
    repositories {
        jcenter()
        mavenCentral()
      }
      dependencies {
        classpath 'org.ysb33r.gradle:vfs-gradle-plugin:1.0.1'
        classpath 'commons-net:commons-net:3.+'  // If you want to use ftp
        classpath 'commons-httpclient:commons-httpclient:3.1' // If you want http/https
        classpath 'com.jcraft:jsch:0.1.48'  // If you want sftp
      }
}
apply plugin : 'org.ysb33r.vfs'

// Create a VFS task
task copyReadme << {
  vfs {
    cp 'https://raw.github.com/ysb33r/groovy-vfs/master/README.md', new File("${buildDir}/tmp/README.md")
  }
}

// it is also possible to update global options for vfs
vfs {
  options {
    http {
      maxTotalConnections 4
    }
  }
}

If you want to see what VFS is going run gradle with --debug

In 1.0 a copy task has been added.

import org.ysb33r.gradle.vfs.tasks.VfsCopy

task download ( type : VfsCopy ) {

    from 'http://somewhere.example/file' (1)

    from URL_2

    // Set the destination root URI. Files will be copied into that folder
    into URL_ROOT

    // Set options for all of the copy operations.
    // These override global options, but are only applicable for source and destination URLs within this task
    options {
      ftp {
        passiveMode true
      }
    }
}
  1. Copy the file at this URI <2>

Any local source URIs will get reflected as an input file in the TaskInputs, otherwise it is just an input If the destination URI is local, it will get reflected as TaskOutputs as a file

Adding extra plugins

From v1.0 onwards additional plugins can be loaded via a new extend block. For more details see this gist: https://gist.github.com/ysb33r/9916940

SMB provider

Download', link=

A provider for accessing SMB shares is now avavilable. The plugin must be loaded separately.

@Grab( 'org.ysb33r.groovy:groovy-vfs-smb-provider:1.0-beta1' ),
@Grab( 'jcifs:jcifs:1.3.17' ),

vfs {
  extend {
    provider className: 'org.ysb33r.groovy.vfsplugin.smb.SmbFileProvider', schemes: ['smb','cifs']
  }

  cp 'smb://someserver/share/dir/file', new File('localfile.txt)
}

NOTE: when embedding windows credentials in the URL use %5C in place of backslash i.e.

  smb://DOMAIN%5cUSERNAME:PASSWORD@HOSTNAME/SHARE/PATH
Note
This provider has a concurrency bug (#73).

S3 provider (EXPERIMENTAL)

Download', link=

A provider for accessing S3 shares is now available and will be fully supported in future version. The plugin must be loaded separately.

@Grab( 'org.ysb33r.groovy:groovy-vfs-cloud-core:0.1-beta1' ),
@Grab( 'org.apache.jclouds:jclouds-all:1.7.2' )
@Grab( 'org.apache.jclouds.driver:jclouds-jsch:1.7.2' )
@Grab( 'org.apache.jclouds.provider:aws-s3:1.7.2'
vfs {
  extend {
    provider className: 'org.ysb33r.groovy.vfsplugin.cloud.s3.S3FileProvider', schemes: ['s3']
  }

  cp 'smb://id:key@bucket/dir/file', new File('localfile.txt)
}

NOTE: Although S3 does not actually support folders, this is simulated through the use of folder names containing / characters.

Command-line utility

Download', link=

A command-line utility mimicking a number of GNU shell utilities is available.

Documentation

Credits

It is seldom that these kind of libraries happen in isolation. It is therefore prudent that I acknowledge the inputs of others in the creation of groovy-vfs

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.