swift-lang / swift-k Goto Github PK

License: Apache License 2.0

Shell 4.60% Python 0.75% Perl 2.43% Makefile 0.21% HTML 1.06% Java 75.10% C++ 2.45% C 0.78% Tcl 0.13% Swift 2.01% Gnuplot 0.12% CSS 0.82% JavaScript 9.05% GAP 0.40% Batchfile 0.05% M4 0.05%

swift-k's Introduction

Swift Parallel Scripting Language

See http://swift-lang.org for documentation, downloads, and general information about the Swift parallel scripting language.

Official releases are available at http://swift-lang.org/downloads.

This is the source code for the Swift/K implementation of the language. Swift/K focuses on robust distributed execution of tasks, particularly command-line programs operating on files, on varied compute resources including clouds and clusters. The sibling Swift/T implementation (https://github.com/swift-lang/swift-t) focuses on high-performance computation on clusters and supercomputers.

Building

Run ant redist to build Swift. The compiled Swift distribution will be created under dist/swift-<version> (dist/swift-svn for the development version). Copy this directory to your preferred location to install it. If you are unfamiliar with Swift, we suggest you start with the Swift tutorials to get yourself familiarized: http://swift-lang.org/docs/

Repository Layout

At the root of the repository are a number of subdirectories that contain the source code and supporting files for the Swift language. The main directories are:

bin: Swift executables
build: working directory for temporary build files
cogkit: modules based on the Java CoG kit
dist: destination for final compiled distribution of Swift
docs: Swift documentation
etc: miscellaneous supporting files
lib: libraries required for Swift, particularly Java jars.
libexec: other libraries required for Swift, particularly Karajan libraries and shell scripts
resources: miscellaneous resources used in building Swift.
src: Java source code for Swift compiler and runtime.
tests: test suite for Swift

Swift depends on a number of additional modules that reside in their own directories under cogkit/modules, for example cogkit/modules/karajan or cogkit/modules/provider-coaster.

Some of the more interesting modules are:

karajan: the Karajan language, used as a compilation target for Swift.
provider-coaster: the Coaster service and Java client for lightweight execution of jobs on remote resources
provider-coaster-c-client: a C/C++ client for the Coaster service
provider-...: other providers that support remote job execution on various computing resources

Java CoG Kit

This distribution of Swift incorporates code originally developed by the CoG (Commodity Grid) project. For information about the Java CoG Kit see README.txt, CHANGES.txt and LICENSE.txt in cogkit, and http://www.cogkit.org/. Thanks to Gregor von Laszweski and the rest of the CoG developers.

swift-k's People

Contributors

Stargazers

Watchers

swift-k's Issues

error in install

$ git clone https://github.com/swift-lang/swift-k.git swift-0.96
$ cd swift-0.96
$ ant redist
Buildfile: build.xml

cleanGenerated:

BUILD FAILED
/home/python/sf_box/swift-0.96/build.xml:415: Directory does not exist:/home/python/sf_box/swift-0.96/src/org/griphyn/vdl/model

Total time: 0 seconds src/org/griphyn/vdl/model

Do I need make new model folder ?

Option -sitelist requires a script

This looks like a bug to me, because I do not see why -sitelist would requires a script to work. Here is the output:

$ swift -sitelist
Swift 0.96.1 git-rev: cdfedcd001993aee741777502c3367cb7da24b10 heads/release-0.96-swift 6281
No Swift script specified
make: *** [show-sites] Error 1

Best,
Aurelien

quickstart documentation problem for new users

In http://swift-lang.org/guides/release-0.96/quickstart/quickstart.html, there is a line
export PATH=$PATH:/path/to/swift-0.96.2/bin

Some new users don't realize that they shouldn't type this directly, but need to replace "/path/to" with the path to their swift. The documentation should state this.

null pointer dereference executing trivial program with release-0.96-swift branch

0.95 doesn't have this problem.

$ cat borked.swift
type file;
(file out_f) foo (file in_f)
{
  out_f = in_f;
}
$ swift borked.swift
Swift 0.96-RC1 git-rev: eac769b7580b1c7d108a20923af605915da9d5ee heads/release-0.96-swift 6273
RunID: run020
Progress: Wed, 04 Feb 2015 16:38:43-0500
Progress: Wed, 04 Feb 2015 16:38:44-0500  Stage out:1

Execution failed:
java.lang.NullPointerException
        assignment @ borked, line: 5
Caused by: java.lang.NullPointerException
        at org.globus.cog.abstraction.impl.common.task.ServiceContactImpl.parse(ServiceContactImpl.java:107)
        at org.globus.cog.abstraction.impl.common.task.ServiceContactImpl.<init>(ServiceContactImpl.java:46)
        at org.griphyn.vdl.karajan.lib.FileCopier.<init>(FileCopier.java:60)
        at org.griphyn.vdl.karajan.lib.SetFieldValue.copyNonComposite(SetFieldValue.java:344)
        at org.griphyn.vdl.karajan.lib.SetFieldValue.deepCopy(SetFieldValue.java:260)
        at org.griphyn.vdl.karajan.lib.SetFieldValue.deepCopy(SetFieldValue.java:233)
        at org.griphyn.vdl.karajan.lib.SetFieldValue.function(SetFieldValue.java:174)
        at org.griphyn.vdl.karajan.lib.SwiftFunction.runBody(SwiftFunction.java:77)
        at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:175)
        at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:110)
        at org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:41)
        at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:110)
        at org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:41)
        at org.globus.cog.karajan.compiled.nodes.user.Function.runBody(Function.java:117)
        at org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.runBody(InvocationWrapper.java:286)
        at org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:89)
        at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:110)
        at org.globus.cog.karajan.compiled.nodes.user.CBFInvocationWrapper.run(CBFInvocationWrapper.java:97)
        at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:110)
        at org.globus.cog.karajan.compiled.nodes.user.InvocationWrapper.run(InvocationWrapper.java:86)
        at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:110)
        at org.globus.cog.karajan.compiled.nodes.Sequential.run(Sequential.java:41)
        at org.globus.cog.karajan.compiled.nodes.restartLog.RestartLog.runBody(RestartLog.java:126)
        at org.globus.cog.karajan.compiled.nodes.InternalFunction.run(InternalFunction.java:175)
        at org.globus.cog.karajan.compiled.nodes.CompoundNode.runChild(CompoundNode.java:110)
        at org.globus.cog.karajan.compiled.nodes.FramedInternalFunction.run(FramedInternalFunction.java:82)
        at k.thr.LWThread.run(LWThread.java:247)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:701)

Can not map stdout to a file

I am using version 0.96 with openmpi 3.1.2, gcc 8.2.0 and openjdk 1.6.
OS is centos-7 with intel haswell xeon processors.
When I try to run the tutorial case I get:
[pnichols@darwin-fe1 part01]$ swift p1.swift
Swift trunk git-rev: 5863615 heads/master 6751
RunID: run008
Progress: Fri, 07 Dec 2018 15:45:40-0700
Execution failed:
Exception in simulation:
Host: localhost
Directory: p1-run008/jobs/s/simulation-hgx1i10o-0
exception @ swift-int-staging.k, line: 187
providerStaging:execute2 @ swift.k, line: 126
executeSelect @ swift.k, line: 193
Caused by: Cannot find executable simulate on site system path
exception @ swift-int-staging.k, line: 182
Caused by: null
Caused by: org.globus.cog.abstraction.impl.common.execution.JobException: Job failed with an exit code of 254
k:assign @ swift.k, line: 157
swift:execute @ p1, line: 31
simulation @ p1, line: 9

It seems that the mappring between stdout and sim.out is not occurring.
Thank you for any advice you can give me on this.
Pat