viktorc / pp4j Goto Github PK
View Code? Open in Web Editor NEWA multiprocessing library for Java that features process pool implementations and a flexible API.
License: Apache License 2.0
A multiprocessing library for Java that features process pool implementations and a flexible API.
License: Apache License 2.0
Hi again,
Firstly, thank you again for helping me on previous topic about JavaProcess.exit() !
Still on that topic, I had another question actually...
While making some tests, I realized that in some rare cases, when there is a crash in C++ part, it may terminates the Java.exe process without throwing any exception to the Java part (I suppose it depends how much the memory is corrupted...).
Problem is that is such situation, I can't throw any exception to cancel the task "Submission" and now the issue I have is coming from this part of code in InternalProcessExecutor.updateSubmissionQueue(InternalSubmission<?> submission):
// If the execute method failed and there was no exception thrown, put the submission back into the queue at the front. submission.setThread(null); submissionQueue.addFirst(submission);
Because of this and because the Submission is not considered as finished, it rerun it again and the process terminates again, and rerun it, etc... forever.
Actually, I am not sure what should be done here, because I understand what was the purpose of this part of code.
However, I think it would help me if I could configure the ProcessExecutor behavior to cancel Submission and raise an exception if for any reason the Process that was running it terminates with an error code.
What do you think ?
Thank you !
Hi,
I am trying to use PP4J as part of a web application deployed on tomcat. I am initializing the JVM pool as part of Spring bean initialization. I see that it gets stuck on the initialization and doesnt allow the application to be deployed. Do you know why that happens?
Can you throw some light on that?
thanks,
Pradeep B.
Hi,
Firstly, congratulation for this great library, it is very helpful. I just had a question about it (not an issue) !
I use this library to call some C++ code and I was wondering if it could be possible to terminate a process in which a crash happened while running a task, so that it won't be used anymore for new tasks.
The goal is to stop using process in which unmanaged C++ memory may be corrupted even if the crash exception was catched correctly.
For ex:
jvmPool.submit((Callable & Serializable) () -> {
SomeCppFunctionCall(); // A CRASH HAPPENS HERE
return rand.nextLong();
}));
Is there a way to "tell" the ProcessPool to terminate the current process in which the crash happened ?
Thank you !
Hi,
Ive built a MatlabExecutor class that starts a static pool as follows:
static {
JavaProcessOptions jvmConfig = new SimpleJavaProcessOptions(JVMArch.BIT_64, JVMType.CLIENT, 128, 256, 256, 5*60*1000);
try {
jvmPool = new JavaProcessPoolExecutor(jvmConfig,4, 8, 1, null, true);
System.out.println("Started jvmPool");
} catch (InterruptedException e) {
logger.error(e,e);
}
}
It will eventually be a servlet, so it has a doPost(request,response) method, but at present it just runs via a test that calls the MatlabExecutor wrapped in a runnable. eg
@Test
public void shouldProcessCalc() throws ServletException, IOException {
for(int x=0;x<16;x++) {
Runnable run = new Runnable() {
@Override
public void run() {
MockHttpServletRequest request = new MockHttpServletRequest();
request.setContent(requestJson.getBytes());
MockHttpServletResponse response = new MockHttpServletResponse();
try {
new ModelScoreExecutor().doPost(request, response);
logger.debug("Response Status: {}",response.getStatus());
String responseStr = response.getContentAsString();
logger.debug("Response Content: {}",responseStr);
} catch (ServletException | IOException e) {
logger.error(e,e);
}
}
};
new Thread(run).start();
}
}
the important part of the doPost method is:
String input = IOUtils.toString(request.getReader());
Json json = Json.read(input);
java.util.concurrent.Future<String> future = jvmPool.submit(new Calc(json),true);
String out = future.get(1, TimeUnit.MINUTES);
response.setStatus(HttpServletResponse.SC_OK);
Calc.java is:
public String call() throws Exception {
String p1 = json.at("rhs").at(0).asString();
String p2 = json.at("rhs").at(1).asString();
List<Object> p = MatlabUtil.getModelScoreInputFromRhs(json);
//makes a JNI call to native matlab calculation
double[][] s = MatlabUtil.executeModelScore(new CPFunctions_MCR.ModelScore(), p1, p2, p);
return MatlabUtil.convert2Lhs(s);
}
So now the problems.
ERROR com.cp.matlab.servlet.MatlabExecutor.doPost(MatlabExecutor.java:141) - java.util.concurrent.ExecutionException: java.io.StreamCorruptedException: invalid type code: E0
java.util.concurrent.ExecutionException: java.io.StreamCorruptedException: invalid type code: E0
at net.viktorc.pp4j.impl.JavaProcessPoolExecutor$JavaSubmission.getResult(JavaProcessPoolExecutor.java:452) ~[pp4j-2.2.jar:?]
at net.viktorc.pp4j.impl.JavaProcessPoolExecutor$JavaSubmission.getResult(JavaProcessPoolExecutor.java:409) ~[pp4j-2.2.jar:?]
at net.viktorc.pp4j.impl.ProcessPoolExecutor$InternalSubmission.getResult(ProcessPoolExecutor.java:471) ~[pp4j-2.2.jar:?]
at net.viktorc.pp4j.impl.ProcessPoolExecutor$InternalSubmissionFuture.get(ProcessPoolExecutor.java:569) ~[pp4j-2.2.jar:?]
at net.viktorc.pp4j.impl.JavaProcessPoolExecutor$CastFuture.get(JavaProcessPoolExecutor.java:502) ~[pp4j-2.2.jar:?]
at com.cp.matlab.servlet.MatlabExecutor.doPost(MatlabExecutor.java:130) [classes/:?]
at com.cp.matlab.servlet.CalcExecutorTest$1.run(CalcExecutorTest.java:65) [test-classes/:?]
Its possible there is a better way to do this, I'm open to suggestions.
I am running my application on a jar, and the classPath (jar:file:/xx.jar!/BOOT-INF/classes!/) is generating a java.nio.file.FileSystemNotFoundException
Hi,
is it possible to include modules / module paths in created java processes?
I'm trying to spawn JavaFx Applications in their own processes to encapsulate them, but have a small GUI just to give a visual indicator of theri progress. But since JavaFX ist now separated, it needs to be added as modules... Without that, the ClassLoader can't find the JavaFX Application
-class...
See here for an example: https://blog.idrsolutions.com/using-javafx-with-java-11/
When I create a simple JavaProcessPool new JavaProcessPoolExecutor(new JavaProcessManagerFactory<>(new SimpleJavaProcessConfig()), ... )
, the classpath of the calling application will be in the child process as well, but this isn't true for modules.
I have outputted the module path in the calling app an in the child app (System.getProperty("jdk.module.path")
) - for the calling, it contains javaFx, for the child it's just null
.
It would be great, if the modules from the calling app could be added to the child process / to the child process's module path (–module-path ...
and –add-modules ...
). Or is there any workaround to get this happening?
Any thoughts? :-)
For me, one reason to use separate processes, is to be able to terminate them, for example if they are running too long or aren't responding anymore. The child-processe's code isn't always behaving wrt. thread interrupts, so I need to have the big gun and kill the process.
In PP4J it seems to be possible in AbstractProcessExecutor, but for the JavaProcessPool, I don't see any chance to get there... Cancelling the returned Future doesn't have any effect on the process.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.