Git Product home page Git Product logo

zio-process's Introduction

ZIO Process

ZIO Process is a simple ZIO library for interacting with external processes and command-line programs.

Production Ready CI Badge Sonatype Releases Sonatype Snapshots javadoc ZIO Process

Introduction

ZIO Process is backed by ZIO Streams, enabling you to work with processes that output gigabytes of data without worrying about exceeding memory constraints.

ZIO Process provides a principled way to call out to external programs from within a ZIO application while leveraging ZIO's capabilities like interruptions and offloading blocking operations to a separate thread pool. We don't need to worry about avoiding these common pitfalls as we would if we were to use Java's ProcessBuilder or the scala.sys.process API since it is already taken care of for you.

Key features of the ZIO Process:

  • Deep ZIO Integration — Leverages ZIO to handle interruption and offload blocking operations.
  • ZIO Streams — ZIO Process is backed by ZIO Streams, which enables us to obtain the command output as streams of bytes or lines. So we can work with processes that output gigabytes of data without worrying about exceeding memory constraints.
  • Descriptive Errors — In case of command failure, it has a descriptive category of errors.
  • Piping — It has a simple DSL for piping the output of one command as the input of another.
  • Blocking Operations

Installation

In order to use this library, we need to add the following line in our build.sbt file:

libraryDependencies += "dev.zio" %% "zio-process" % "0.7.2" 

Native support

Some features might have a different behaviour when using Scala Native. Creating non-existent commands do not throw a corresponding error. bash command might be needed when executing a script. In some cases, using ZStreams as standard input might block the process. Instead, a Java InputStream can be used to write to the standard input of the process.

Example

Here is a simple example of using ZIO Process:

import zio._
import zio.process.Command

import java.io.File

object ZIOProcessExample extends ZIOAppDefault {
  val myApp = for {
    fiber <- Command("dmesg", "--follow").linesStream
      .foreach(Console.printLine(_))
      .fork
    cpuModel <- (Command("cat", "/proc/cpuinfo") |
      Command("grep", "model name") |
      Command("head", "-n", "1") |
      Command("cut", "-d", ":", "-f", "2")).string
    _ <- Console.printLine(s"CPU Model: $cpuModel")
    _ <- (Command("pg_dump", "my_database") > new File("dump.sql")).exitCode
    _ <- fiber.join
  } yield ()

  override def run = myApp
}

Documentation

Learn more on the ZIO Process homepage!

Contributing

For the general guidelines, see ZIO contributor's guide.

Code of Conduct

See the Code of Conduct

Support

Come chat with us on Badge-Discord.

License

License

zio-process's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zio-process's Issues

Scala.js support

I'm not sure what this will look like, but I believe it makes sense to support Scala.js such as when running in the context of Node. Will need to be investigated.

Consider adding an error ADT

This came up in the Discord. For context, I'm pasting my response:

I'm not against creating an error ADT for zio-process. There was no real opinionated reason for making it Throwable. The only real reason it's like that currently is because that's kind of the go-to default when wrapping Java libraries unless you dig deeper into the internals and map out all the ways it can fail. Sometimes that's easy enough to do... but other times it can be quite the task.

I agree that the way it is currently that Throwable is kind of exposing the fact that it's wrapping a Java library. That sort of feels like an implementation detail leaking out. If it were a library written 100% from scratch I wouldn't expect it to be Throwable.

I can understand the frustration with Throwable. Nobody wants to do control-flow logic on Throwable.getMessage (if the exceptions in the Java library aren't fine-grained enough) and so on.

So we would need to look into all the possible ways the Java lib can fail and design a reasonable ADT that captures those failures.

Introducing type parameters

I haven't thought about this too much yet, but we might be able to introduce type parameters for input and output. Maybe something like Command[-I, +O]? And then provide evidence where it makes sense (like for "this operation only makes sense on List[String] and so on).

That's what I got from when @jdegoes briefly mentioned it, but please correct me if I misinterpreted something.

Provide way to also kill descendants

I'm trying to migrate my own ProcessBuilder code to this library and need a way to not only kill the process, but also its decendants (because the process doesn't properly clean them up).

Currently, I have code like this in my codebase:

zio.blocking
          .effectBlockingCancelable(job.run(opt))(
            (for {
              _ <- (ZIO.debug("trying to interrupt the GAMS process nicely") *> ZIO
                .effect(job.interrupt())).repeat(Schedule.recurs(3) && Schedule.spaced(2.seconds)).ignore
              _ <- (ZIO.debug("trying to stop more nicely now.. ") *> ZIO.effect(job.tryNicely())).ignore *> ZIO.sleep(
                5.seconds)
              _ <- (ZIO.debug("going full rampage... ") *> ZIO.effect(job.kill()).ignore)
            } yield ()).provide(clock)

with the following helper methods:

def tryNicely() = {
    println("trying nicely to destroy the process")
    val process     = processRef.get()
    val descendants = process.descendants().iterator().asScala.toList
    descendants.foreach { processHandle =>
      processHandle.destroy()
    }
    processRef.get().destroy()
  }      
    
    ...
  def kill(): Unit = {
    val proc        = processRef.get()
    val descendants = proc.descendants().iterator().asScala.toList
    if (proc.isAlive || descendants.exists(_.isAlive)) {
      println("Process or children are still alive, destroying forcibly")
      proc.descendants().iterator().asScala.foreach(processHandle => processHandle.destroyForcibly())
      processRef.get().destroyForcibly()
    }
  }

maybe we can incorporate this behaviour or a simple way to achieve the same into this lib?

I tried to create my own exitCode method, but the CommandThrowable object is package private to process

      def exitCodeDestroying: Process => ZIO[Blocking, CommandError, ExitCode]              =
        process => effectBlockingCancelable(ExitCode(process.waitFor()))(UIO({
          process.descendants().forEach(p => { p.destroy(); () })
          process.destroy()
        })).refineOrDie {
          case CommandThrowable.IOError(e) => e
        }

hi

test for deletable

Scala Native support

This may not actually require any code changes. From what I can tell, Scala Native already supports Java's ProcessBuilder interface (which is what ZIO Process is based on).

It's a matter of trying it out and seeing where Scala Native support for ZIO is. I believe it's still a work in progress but works to some degree.

some processes started by zio-process v0.7.0 left running after zio program completes

If I run this:

    def program = for {
      hello <-
          Command("yes", "Hello")
          .linesStream
          .debug("inside `yes`")
          .runHead
          .someOrElse("fallback")
      echo  <- Command("echo", hello).string
      _     <- zio.Console.printLine(s"final echo: ${echo}")
    } yield ()

    program *> ZIO.never

I see the following output:

inside `yes`: Hello
final echo: Hello

but then I also see the yes process still running:

$> ps aux | grep yes
249541  0.0  0.0  10608   520 ?        S    11:45   0:00 yes Hello

It stops when I stop jvm process.

This doesn't happen when I change Command("yes", "Hello") to Command("echo", "Hello") or Command("ls", ".").

It still happens when I change Command("yes", "Hello") to (Command("yes", "Hello") | Command("head", "-n", "1"))

Tested on v0.7.0 release.

Make an io on getting int exitCode

https://zio.github.io/zio-process/docs/overview/overview_basics#exit-code
ExitCode all mapped to 0, 1 conceptually (by zio.ExitCode).
This is a convenient design for chaining things up, and one other motivation behind this could be thought as a java nature, on the exiting code.
On the other hand, in some cases a process's real exitCode matters for recovery of sub process failures.
So, An io on getting the real exitCode such as int value 1 2 ... is quite crucial for uses to verse some real scripts in zio-process.

Release new version

A new version release would be helpful to implement full native and scalajs support for projects with zio-process as dependency.

Scaladoc page/link improvements

Right now the link to the scaladocs page is a little strange:

Migrate to GitHub Actions

Since most of the other projects have migrated already. We can use the zio project itself as a reference for the config.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.