Git Product home page Git Product logo

play-parseq's Introduction

Play-ParSeq

Build Status

Play-ParSeq is a Play module which seamlessly integrates ParSeq with Play Framework.

ParSeq is a Java framework for writing async code, which has several advantages over Java CompletionStage or Scala Future, e.g. ParSeq Trace for async code's runtime visualization, async code reuse via Task composition and taking control of async code's lifecycle.

Key features:

  • Executes ParSeq Task and generates Java CompletionStage or Scala Future, which allows ParSeq Tasks to be used for executing Play Action.
  • Converts from Java Callable<CompletionStage<T>> or Scala () => Future[T] to ParSeq Task, which allows existing code using Play native APIs to be integrated with ParSeq Tasks.
  • Supports ParSeq Trace.
  • Provides both Scala and Java API.
  • Requires Play 2.6.

Releasing

Every change on master branch, for example a merged pull request, lands a new version in Bintray's JCenter and Maven Central. This way we continuously deliver improvements in small batches, ensuring quality and compatibility. Check out latest release notes! Release automation is handled by Shipkit (http://shipkit.org) and configured in shipkit.gradle.

Quick start

Core Java

  1. Put the preset module PlayParSeqModule into your application.conf.

    ...
    play.modules.enabled += "com.linkedin.playparseq.j.modules.PlayParSeqModule"
    ...
    
  2. Inject PlayParSeq into your Controller.

    ...
    private final PlayParSeq _playParSeq;
    @Inject
    public Sample(final PlayParSeq playParSeq) {
        _playParSeq = playParSeq;
    ...
  3. Use PlayParSeq in your Action.

    ...
    public CompletionStage<Result> demo() {
        // Convert to ParSeq Task
        Task<String> helloworldTask = _playParSeq.toTask("helloworld", () -> CompletableFuture.completedFuture("Hello World"));
        // Run the Task
        return _playParSeq.runTask(Http.Context.current(), helloworldTask).thenApply(Results::ok);
    }
    ...

Enable ParSeq Trace Java

  1. Put additional preset module for ParSeqTraceModule into your application.conf.

    ...
    play.modules.enabled += "com.linkedin.playparseq.trace.j.modules.ParSeqTraceModule"
    ...
    
  2. Put ParSeq Trace resource route into your routes.

    ...
    ->         /                       com.linkedin.playparseq.trace.Routes
    ...
    
  3. Annotate your Action by putting @With(ParSeqTraceAction.class).

    ...
    @With(ParSeqTraceAction.class)
    public CompletionStage<Result> demo() {
    ...
  4. Access [original-route]?parseq-trace=true will display ParSeq Trace Viewer for your original request if your application is in dev mode.

Core Scala

  1. Put the preset module PlayParSeqModule into your application.conf.

    ...
    play.modules.enabled += "com.linkedin.playparseq.s.modules.PlayParSeqModule"
    ...
    
  2. Inject PlayParSeq into your Controller.

    ...
    class Sample @Inject()(playParSeq: PlayParSeq, cc: ControllerComponents) extends AbstractController(cc) {
    ...
  3. Use PlayParSeq in your Action.

    ...
    def demo = Action.async(implicit request => {
        // Convert to ParSeq Task
        val helloworldTask = playParSeq.toTask("helloworld", () => Future("Hello World"))
        // Run the Task
        playParSeq.runTask(helloworldTask)
            .map(Ok(_))
    })
    ...

Enable ParSeq Trace Scala

  1. Put additional preset module for ParSeqTraceModule into your application.conf.

    ...
    play.modules.enabled += "com.linkedin.playparseq.trace.s.modules.ParSeqTraceModule"
    ...
    
  2. Put ParSeq Trace resource route into your routes.

    ...
    ->         /                       com.linkedin.playparseq.trace.Routes
    ...
    
  3. Inject ParSeqTraceAction into your Controller.

    ...
    class Sample @Inject()(playParSeq: PlayParSeq, parSeqTraceAction: ParSeqTraceAction, cc: ControllerComponents) extends AbstractController(cc) {
    ...
  4. Use parSeqTraceAction.async for your Action.

    ...
    def demo = parSeqTraceAction.async(implicit request => {
    ...
  5. Access [original-route]?parseq-trace=true will display ParSeq Trace Viewer for your original request if your application is in dev mode.

More examples

Please see /sample.

FAQ

Why is there no ParSeq Trace Viewer even though I've added parseq-trace=true to my query?

A: Please first make sure you annotated your Action with @With(ParSeqTraceAction.class) in Java, or you used parSeqTraceAction.async for your Action in Scala. Then check whether you meet all ParSeqTraceSensor requirements of showing ParSeq Trace. And also don't forget using runTask in your Action. Please also note that ParSeq Trace Viewer will be blocked by strict Content Security Policy rules because of some inline scripts and styles.

How can I use my own module settings?

A: If you don't want to use the preset modules, please write your own Module and register it by putting into your application.conf. Almost every part of Play-ParSeq follows the DI, so you can easily replace any part with your own.

Are the preset modules configurable?

A: Yes. If you don't want to use the default value, please insert the corresponding settings into your application.conf.

Name Description Default
parseq.engine.numThreads The number of threads in Engine's pool. Available processors + 1
parseq.engine.terminationWaitSeconds The maximum time to wait for Engine's termination in the unit of seconds. 1
parseq.trace.docLocation The file path of the dot, which is part of graphviz for generating Task's graphviz view. Registered location if installed
parseq.trace.cacheSize The number of cache items in GraphvizEngine. 1024
parseq.trace.getTimeoutMilliseconds The timeout of the GraphvizEngine execution in the unit of milliseconds. 5000
parseq.trace.parallelLevel The maximum of the GraphvizEngine's parallel level. 1
parseq.trace.delayMilliseconds The delay time between different executions of the GraphvizEngine in the unit of milliseconds. 5
parseq.trace.processQueueSize The size of the GraphvizEngine's process queue. 1000

Can I run multiple ParSeq Tasks in one request?

A: Yes. Play-ParSeq supports this. However, you shouldn't be running multiple Tasks, otherwise the order of execution might not be accurate, which minimizes the benefits of ParSeq.

Does ParSeq Trace support streaming?

A: Yes.

How can I replace the requirements of showing ParSeq Trace?

A: You can follow the instructions below:

  1. Implements ParSeqTraceSensor with your own requirements.

    ...
    // Java
    @Singleton
    public class MySensorImpl implements ParSeqTraceSensor {
        @Override
        public boolean isEnabled(final Http.Context context, final ParSeqTaskStore parSeqTaskStore) {
            return [your-requirements];
        }
    }
    ...
    ...
    // Scala
    @Singleton
    class MySensorImpl extends ParSeqTraceSensor {
        override def isEnabled(requestHeader: RequestHeader, parSeqTaskStore: ParSeqTaskStore): Boolean =
            [your-requirements]
    }
    ...
  2. Create your own Module to include the binding of your ParSeqTraceSensor.

    ...
    // Java
    public class MyModule extends Module {
        @Override
        public Seq<Binding<?>> bindings(final Environment environment, final Configuration configuration) {
            return seq(
                bind(ParSeqTraceSensor.class).to(MySensorImpl.class),
                [other-bindings]);
        }
    }
    ...
    ...
    // Scala
    class MyModule extends Module {
        override def bindings(environment: Environment, configuration: Configuration): Seq[Binding[_]] = Seq(
            bind[ParSeqTraceSensor].to[MySensorImpl],
            [other-bindings])
    }
    ...
  3. Register your Module in your application.conf.

    ...
    play.modules.enabled += [package-of-MyModule]
    ...
    
  4. Have two ice-creams.

Can I put in the Java version of ParSeqTaskStore and then get from the Scala version?

A: It's not possible with the current ParSeqTaskStore implementation, which is based on Play Framework's Java or Scala specific request APIs. Play 3.0 will hopefully provide a common underlying request which will remove this limitation. However, you can inject your own implementation of ParSeqTaskStore, such as shared cache or local file, to make this happen.

License

Copyright 2015 LinkedIn Corp.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

play-parseq's People

Contributors

bbarkley avatar benmccann avatar chenfanggm avatar franklinyinanding avatar miracle2121 avatar mockitoguy avatar shipkit-org avatar syzboy avatar zackthehuman avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

play-parseq's Issues

Not sync with Maven Central

The artifacts are not showing in Maven Central. Contacted with Sonatype, they are not seeing traces on oss.sonatype.org. See here.

Enable Travis CI job

Once we get admin rights to the project, let's setup Travis CI. I will provide travis config in PR #10 but it would be great if someone from the core team enabled Travis CI. This way you'll learning how to use Travis!

Publish to Maven Central

Now we've successfully published to Bintray. We need to include the project with JCenter then sync with Maven Central.

Enable cross-building for Scala 2.13

Hi folks, I have a Play application that is on Scala 2.13 and I'm not able to use play-parseq because it only builds artifacts for 2.11 and 2.12. It appears that the 2.12 artifact is not compatible with 2.13 due to differences in how scala.Serializable is implemented (there are likely more issues, that's just one example).

I'm happy to send a PR if we're open to expanding the cross-building versions to include 2.13.

Create 'com.linkedin.play-parseq' project in Maven Central

To start publishing to Maven Central we need to follow the official guide:

http://central.sonatype.org/pages/ossrh-guide.html

@FranklinYinanDing, it's best if you or one of the team members do it. This way you'll learn the process and can manage/maintain it long term. I can help. The link above should get you a good start. Once you created project for Maven Central, you can generate key+password and we will use to configure Shipkit to automatically publish to Maven Central.

Not fully imported in JCenter

After successfully released, the artifacts are not fully imported in JCenter. Only one artifact play-parseq_2.12 got imported. See here.

Travis CI timeout when publishing

Travis CI timed out when publishing.

@FranklinYinanDing, can you make following local test on clean checkout of play-parseq?

export BINTRAY_API_KEY=your-secret
./gradlew bintrayUpload -PdryRun
./gradlew bintrayUpload

Above should work and should publish to LinkedIn Bintray test repo (https://bintray.com/linkedin/test-repo/play-parseq).

Once we get Bintray publications working we can:

  • delete v.0.8.1 tag & push
  • revert v.0.8.1 code push performed by Shipkit & push
  • trigger Travis to build new version

Once we get Travis to automatically publish without problems we can:

  • clean up the repo again (remove tag and automated commit)
  • change the 'test-repo' to 'maven' in shipkit.gradle & push
  • observe how Travis builds new version and ships to JCenter :D

Then, let's work on Maven Central releases.

Fully automated releases

Problem statement

Let's automate the releases of play-parseq. Goals:

  • releases to Maven Central to streamline consumption by the community (and to create more engaged community)
  • release notes generation
  • example LinkedIn OS project that has full release automation

Solution options

  1. Automate SBT build
  2. Migrate to Gradle and automate with Gradle plugins
  3. Add Gradle for releases, delegate to SBT

Discussion (with @FranklinYinanDing):

  • ad1) SBT is a dead. At LinkedIn we actively migrate to Gradle. SBT automation will duplicate Gradle automation we use in other projects.
  • ad2) Migrate to Gradle is ideal but adds extra effort.
  • ad3) Good pragmatic approach. We can get the releases automated without committing to full SBT->Gradle migration. We demonstrate how Gradle can wrap foreign build tool to automate releases. Risk exists that Gradle-SBT layer gets complicated and hard to maintain (if that happens we'll do option 2).

Implementation

I suggest to start using Shipkit Gradle plugins (http://shipkit.org) that Mockito community leverages to implement automated releases.

Suggested design

  • Enable Travis CI
  • Use and configure Shipkit Gradle plugins
    • Gradle will delegate to SBT to run build
    • Every PR merged to master will produce a new version in Maven Central
    • Documentation and release notes are automatically generated

Milestone 1

Automated releases to JCenter (very visible public repo)

Milestone 2

Automated releases to Maven Central, easy local testing.

  • Convenient local testing (easy way to build local snapshot) - @mockitoguy
  • Maven Central releases
    • - request Maven Central releases for "com.linkedIn.play-parseq" libraries - #18
  • LinkedIn Artifactory releases (optional)

Thoughts / feedback?

Exciting!!! Let's get the releases fully automated and provide an example for other LinkedIn projects ๐Ÿ—ก

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.