Git Product home page Git Product logo

spring-batch-rest's Introduction

spring-batch-rest

Maven Central Javadocs Build Status Coverage Status Maintainability

REST API for Spring Batch based on Spring Boot 2.2 and Spring HATOEAS. It comes with an OpenAPI 3 documentation provided by Springdoc.

Supports Java 8 and above. Tested on OpenJDK 8 and 11.

Features

  • Get information on jobs, job executions, and Quartz schedules
  • Start job execution (synchronous or asynchronous) with optional job property overrides. The job properties can either be obtained via a custom API or via standard Spring Batch job parameters, accessible from step-scoped beans.

Getting Started

To integrate the REST API in your Spring Boot project, first ensure you have an entry point to your application such as

@SpringBootApplication
public class SpringBootApp {
    public static void main(String[] args) {
        SpringApplication.run(SpringBootApp.class, args);
    }
}

Then, simply add one of the following two dependencies to your project:

Core API

The spring-batch-rest-api dependency comes with jobs and jobExecutions REST endpoints. It is recommended if you don't require Quartz for scheduling your jobs.

Maven:

<dependency>
    <groupId>com.github.chrisgleissner</groupId>
    <artifactId>spring-batch-rest-api</artifactId>
    <version>1.5.1</version>
</dependency>

Gradle:

implementation 'com.github.chrisgleissner:spring-batch-rest-api:1.5.1'

Quartz API

The spring-batch-rest-quartz-api dependency includes everything above and and additionally exposes Quartz schedules via the jobDetails REST endpoint.

Maven:

<dependency>
    <groupId>com.github.chrisgleissner</groupId>
    <artifactId>spring-batch-rest-quartz-api</artifactId>
    <version>1.5.1</version>
</dependency>

Gradle:

implementation 'com.github.chrisgleissner:spring-batch-rest-quartz-api:1.5.1'

See it in Action

To see spring-batch-rest-api in action, first install Java 8 or 11 as well as Maven, then run

mvn install -Dmaven.test.skip; java -jar example/api/target/*.jar

For spring-batch-rest-quartz-api, run

mvn install -Dmaven.test.skip; java -jar example/quartz-api/target/*.jar

Once it's up, check the Swagger REST API docs at http://localhost:8080/swagger-ui.html.

Sample Walkthrough

Here's how to run a sample job, assuming you've started the API as described above:

  • Click on POST to the left of /jobExecutions, then on Try it Out on the right-hand side. Replace the contents of the Request body with {"name": "personJob"}, then click Execute.
  • The job has now been triggered. When it completes, you'll get a response body similar to:
{
  "jobExecution": {
    "id": 0,
    "jobId": 0,
    "jobName": "personJob",
    "startTime": "2023-09-01T19:01:55.264",
    "endTime": "2023-09-01T19:01:55.317",
    "exitCode": "COMPLETED",
    "exitDescription": "",
    "status": "COMPLETED",
    "exceptions": []
  },
  "_links": {
    "self": {
      "href": "http://localhost:8080/jobExecutions/0"
    }
  }
}
  • You can now view this and other recently completed jobs by clicking on GET to the left of /jobExecutions, then on Try it Out followed by Execute. You should see something like:
{
  "_embedded": {
    "jobExecutionResourceList": [
      {
        "jobExecution": {
          "id": 0,
          "jobId": 0,
          "jobName": "personJob",
          "startTime": "2023-09-01T19:01:55.264",
          "endTime": "2023-09-01T19:01:55.317",
          "exitCode": "COMPLETED",
          "exitDescription": "",
          "status": "COMPLETED",
          "exceptions": []
        },
        "_links": {
          "self": {
            "href": "http://localhost:8080/jobExecutions/0"
          }
        }
      }
    ]
  },
  "_links": {
    "self": {
      "href": "http://localhost:8080/jobExecutions?limitPerJob=3"
    }
  }
}

REST Endpoints

The following REST endpoints are available:

Jobs

HTTP Method Path Description
GET /jobs All jobs
GET /jobs/{jobName} Single job

Job Executions

HTTP Method Path Description
GET /jobExecutions Latest 3 executions for each job, sorted by descending end time (or start time if still running)
GET /jobExecutions/{id} Single job execution
POST /jobExecutions Start job execution with optional property overrides

Request Parameters for GET /jobExecutions

Parameter Default Value Description
jobName empty Regular expression of the job names to consider. If empty, all job names are used.
exitCode empty Exit code of the job execution. Can be COMPLETED, EXECUTING, FAILED, NOOP, STOPPED or UNKNOWN as per ExitStatus. If empty, all exit codes are used.
limitPerJob 3 Maximum number of job executions to return for each job.

Examples

HTTP Method Path Description
GET /jobExecutions?limitPerJob=50 Latest 50 executions for each job
GET /jobExecutions?jobName=foo&exitCode=FAILED Latest 3 failed executions for 'foo' job
GET /jobExecutions?jobName=foo.*&exitCode=FAILED&limitPerJob=10 Latest 10 failed executions for jobs with a name starting with 'foo'

Quartz Schedules

As mentioned above, these endpoints are only exposed if you're using the spring-batch-rest-quartz-api dependency:

HTTP Method Path Description
GET /jobDetails All Quartz schedules
GET /jobsDetails/{quartzGroupName}/{quartzJobName} Single Quartz schedule

Error Handling

Where possible, subclasses of the Spring Batch JobExecutionException are mapped to an appropriate HTTP status code and the response body contains further details.

For example, trying to start a nonexistent job results in a response with a 404 status code and the following response body:

{
  "status": "404 NOT_FOUND",
  "message": "No job configuration with the name [foo] was registered",
  "exception": "NoSuchJobException",
  "detail": "Failed to start job 'foo' with JobConfig(name=foo, properties={foo=baz10}, asynchronous=false). Reason: No job configuration with the name [foo] was registered"
}

Configuration

The default behaviour of the REST API can be tweaked via several Spring properties which can be placed in application.properties.

Job Execution Caching

com.github.chrisgleissner.springbatchrest.jobExecutionCacheSize (default: 100)

For performance reasons, /jobExecutions queries are performed against an in-memory cache of recent job executions. If the limitPerJob request parameter is larger than the value of this property, the cache is bypassed and the Spring Batch JobExplorer is used instead.

Large jobExecutionCacheSize values will result in increased heap use, but small values combined with large limitSize request parameters will cause increased REST query latencies. Thus, if you increase this property value, you may also want to increase your heap size.

The cache only contains job executions since the Spring context creation, ie. it is not warmed up from the JobExplorer and the DB this may rely on. If you want to be able to query job executions that were performed earlier, eg. during a prior JVM execution, you may want to disable caching. To do so, simply set the property to 0.

Repeated Job Execution

com.github.chrisgleissner.springbatchrest.addUniqueJobParameter (default: true)

Spring Batch prevents repeated invocations of a job unless you use different properties (aka job parameters) each time. To bypass this, a unique property (ie. a random UUID) is added to each job invocation. You can disable this by setting the property to false.

Disable Spring Batch REST API REST Endpoints

com.github.chrisgleissner.springbatchrest.enabled=false (default: true)

Useful if you only want to expose the REST API in certain environments.

Disable Swagger UI

springdoc.swagger-ui.enabled=false (default: true)

See https://github.com/springdoc/springdoc-openapi for further config options.

Disable Custom Exception Handling

com.github.chrisgleissner.springbatchrest.controllerAdvice=false (default: true)

This disables the global exception handling via com.github.chrisgleissner.springbatchrest.api.core.jobexecution.ResponseExceptionHandler.

Job Property Overrides

Properties can be overridden when starting a job via REST. You can then access these overrides in one of the following ways.

@Value

Annotate your Spring bean method with @StepScope and use the @Value annotation on a method parameter to specify the desired job parameter name.

Please note that this approach won't transparently fall back to Spring environment properties. Thus, if this is desired, you should manually check if a job parameter is null and in this case return it from the Spring Environment instance.

Example:

@Bean @StepScope  
ItemWriter<Object> writer(@Value("#{jobParameters['sampleProperty']}") String sampleProperty) {
    // ... 
}

PropertyResolver

When using AdhocStarter, you can create a Job using a JobBuilder and pass in a Consumer<PropertyResolver>.

Properties looked up from this PropertyResolver transparently fall back to the Spring environment if properties can't be found in the job parameters.

Example:

Job job = jobBuilder.createJob("sampleJob", propertyResolver -> {
    String propertyValue = propertyResolver.getProperty("sampleProperty");
    // ...
});

JobProperties

In case you don't execute the same job concurrently, you may also look up properties from the JobProperties singleton.

Properties looked up from this singleton transparently fall back to the Spring environment if properties can't be found in the job parameters.

This approach is deprecated as it doesn't work with concurrent execution of the same job. Therefore, it is recommended to use one of the other approaches.

Example:

@Bean
ItemWriter<Object> writer() {
    return new ItemWriter<Object>() {
        @Override
        public void write(List<?> items) throws Exception {
           String sampleProperty = JobPropertyResolvers.JobProperties.of("sampleJob").getProperty("sampleProperty");
           // ...
        }
    }
}

Utilities

The util module contains code for registering, starting and scheduling jobs:

JobBuilder builds a simple job based on a Runnable:

Job job = jobBuilder.createJob("jobName", () -> System.out.println("Running job"));

AdHocScheduler registers and triggers a job using a Quartz CRON trigger. This can be performed at run-time rather than Spring wiring time which allows for simplified set-up of a large number of jobs that only differ slightly:

adHocScheduler.schedule("jobName", job, "0/30 * * * * ?");

AdHocStarter is similar to AdHocScheduler, but used for immediately starting a job:

adHocStarter.start(job);

spring-batch-rest's People

Contributors

cg-exp avatar chrisgleissner avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

spring-batch-rest's Issues

Add Parameter Information/Params to Quartz API Endpoints

Hey there, I don't have time to fix this or add the feature so I figured I'd log some things that would be good additions to the project.

Now that JobParameters can be specified for scheduled jobs, it would be pretty straightforward via the structures you've setup to expose these via the API, so you can see the parameters that a scheduled job is configured to run for.

Would also be pretty cool to add an optional parameter/body to specify job parameters for a scheduled job that you're initiating via the REST API.

JobConfig adaptations

Hi,

I want to propose 2 changes in JobConfig class, if possible:

  1. In my project I have to use jackson-core:2.9.9 to work properly with other dependencies. To avoid an exception in that version, JobConfig needs @NoArgsConstructor and @AllArgsConstructor annotations. Could be added?

  2. All the properties in JobConfig are String values (Map<String, String> properties), could you allow other types (i.e Map<String, Object> properties) ?

Basic Authentication

Hi, do you have any intention of setting up some kind of authentication to enter the Swagger page?

Date job parameters provided as string

We'v a job that requires a creationDate as date parameter.

While JobParamUtil checks for date parameters, the issue seems to be that the properties map is never filled with date properties as the deserializer treats them as string by default.

Did I miss any way to force a date or is this feature currently not supported?

500 error on getJobDetails quartz

Hey there, I don't have time to fix this so I figured I'd log some issues I found and some enhancement requests.

Calling /jobDetails/{quartzGroupName}/{quartzJobName}

For a job that doesn't exist returns a 500 error. I found the line it throws in the past, but can't recall - its pretty obvious - I think there is a filter that runs, then it calls .get(0) on the resulting structure which is empty, so it throws.

Also, {quartzGroupName} should now be made optional, and default to null. Internally, null will use the QuartzDefaultGroup, so if unspecified in the API/swagger would default to null=defaultGroup.

Ideally the API would return 404 with a response body indicating if the quartzGroupName was not found or if the quartzJobName was not found.

Disable quartz

Hello,

Is it possible to disable quartz? I just want to execute jobs by api request

ERROR 500 with stacktrace when a job is lauched more than once.

Hello,
when I start a job with the same parameter it cause an error 500.
I think It would be nice not to fail in this case. but catch the error and display it.

Error on the client side.

{
  "timestamp": "2018-12-22T13:54:36.839+0000",
  "status": 500,
  "error": "Internal Server Error",
  "message": "Failed to start personJob",
  "trace": "java.lang.RuntimeException: Failed to start personJob\n\tat com.github.chrisgleissner.springbatchrest.util.adhoc.AdHocStarter.start(AdHocStarter.java:53)\n\tat com.github.chrisgleissner.springbatchrest.api.jobexecution.JobExecutionService.launch(JobExecutionService.java:74)\n...",
 "path": "/jobExecutions"
}

Error on the server

rg.springframework.batch.core.repository.JobInstanceAlreadyCompleteException: A job instance already exists and is complete for parameters={upperCase=true, additionalProp3=stringz, additionalProp2=string}.  If you want to run this job again, change the parameters.
	at org.springframework.batch.core.repository.support.SimpleJobRepository.createJobExecution(SimpleJobRepository.java:132) ~[spring-batch-core-4.1.0.RELEASE.jar:4.1.0.RELEASE]
...

AdHocStarter ".start()" docs update

Looks like AdHocStarter takes a parameter "JobConfig" which is a member class of this library.

Docs state that this start() call takes generic Spring Batch Param types:

String, Job

Any information on how to correctly configure & use the JobConfig as a parameter to call start() would be very much appreciated.

Thanks for any help or clarification!

Question: Quartz Group Name

The Quarz Group Name seems to be hard-coded to "group".

How does one change this?

I assume since its exposed through the rest API there is some way to override it.. but looking at the class I'm sort of dense when it comes to how to do it.

Thanks for any help!

mariadb backend

Hi Chris:

Thanks for your open-source work of spring-batch-rest.

I fork it on the github as a new project and then I make it compatible with spring boot 1.5.3(the current version we used for work). After I plug it in an existed spring batch/boot application with mariadb backend, when I visit http://localhost:8130/jobs, I expect I will get the jobs stored in mariadb(ie: BATCH_JOB_INSTANCE), but I get no one.

Can you please give me some advices to make this work?
Thanks in advance.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.