viktorc / pp4j Goto Github PK

Hi again,

Firstly, thank you again for helping me on previous topic about JavaProcess.exit() !

Still on that topic, I had another question actually...

While making some tests, I realized that in some rare cases, when there is a crash in C++ part, it may terminates the Java.exe process without throwing any exception to the Java part (I suppose it depends how much the memory is corrupted...).

Problem is that is such situation, I can't throw any exception to cancel the task "Submission" and now the issue I have is coming from this part of code in InternalProcessExecutor.updateSubmissionQueue(InternalSubmission<?> submission):
// If the execute method failed and there was no exception thrown, put the submission back into the queue at the front. submission.setThread(null); submissionQueue.addFirst(submission);

Because of this and because the Submission is not considered as finished, it rerun it again and the process terminates again, and rerun it, etc... forever.

Actually, I am not sure what should be done here, because I understand what was the purpose of this part of code.
However, I think it would help me if I could configure the ProcessExecutor behavior to cancel Submission and raise an exception if for any reason the Process that was running it terminates with an error code.

What do you think ?

Thank you !

Hi,

I am trying to use PP4J as part of a web application deployed on tomcat. I am initializing the JVM pool as part of Spring bean initialization. I see that it gets stuck on the initialization and doesnt allow the application to be deployed. Do you know why that happens?

Can you throw some light on that?

thanks,
Pradeep B.

Hi,

Firstly, congratulation for this great library, it is very helpful. I just had a question about it (not an issue) !

I use this library to call some C++ code and I was wondering if it could be possible to terminate a process in which a crash happened while running a task, so that it won't be used anymore for new tasks.
The goal is to stop using process in which unmanaged C++ memory may be corrupted even if the crash exception was catched correctly.

For ex:
jvmPool.submit((Callable & Serializable) () -> {
SomeCppFunctionCall(); // A CRASH HAPPENS HERE
return rand.nextLong();
}));

Is there a way to "tell" the ProcessPool to terminate the current process in which the crash happened ?

Thank you !

Hi,

Ive built a MatlabExecutor class that starts a static pool as follows:

static {
		JavaProcessOptions jvmConfig = new SimpleJavaProcessOptions(JVMArch.BIT_64, JVMType.CLIENT, 128, 256, 256, 5*60*1000);
		
		try {
			jvmPool = new JavaProcessPoolExecutor(jvmConfig,4, 8, 1, null, true);
			System.out.println("Started jvmPool");
		} catch (InterruptedException e) {
			logger.error(e,e);
		}
	}

It will eventually be a servlet, so it has a doPost(request,response) method, but at present it just runs via a test that calls the MatlabExecutor wrapped in a runnable. eg

	@Test
	public void shouldProcessCalc() throws ServletException, IOException {
	
		for(int x=0;x<16;x++) {
			Runnable run = new Runnable() {
				
				@Override
				public void run() {
					MockHttpServletRequest request = new MockHttpServletRequest();
					request.setContent(requestJson.getBytes());
					MockHttpServletResponse response = new MockHttpServletResponse();
					try {
						new ModelScoreExecutor().doPost(request, response);
						logger.debug("Response Status: {}",response.getStatus());
						String responseStr = response.getContentAsString();
						logger.debug("Response Content: {}",responseStr);
					} catch (ServletException | IOException e) {
						logger.error(e,e);
					}
				}
			};
			new Thread(run).start();
		}
}

the important part of the doPost method is:

			String input = IOUtils.toString(request.getReader());
			Json json = Json.read(input);

			java.util.concurrent.Future<String> future =  jvmPool.submit(new Calc(json),true);

			String out = future.get(1, TimeUnit.MINUTES);		
			response.setStatus(HttpServletResponse.SC_OK);
			

Calc.java is:

	public String call() throws Exception {
		String p1 = json.at("rhs").at(0).asString();
		String p2 = json.at("rhs").at(1).asString();

		List<Object> p = MatlabUtil.getModelScoreInputFromRhs(json);
//makes a JNI call to native matlab calculation
		double[][] s = MatlabUtil.executeModelScore(new CPFunctions_MCR.ModelScore(), p1, p2, p);
		return MatlabUtil.convert2Lhs(s);

	}

So now the problems.

1. If I run the above with "jvmPool.submit(new Calc(json), false);" the I get about 8 successful calcs, and then concurrent exceptions and stream corruptions. It seems like the submissions are not truly atomic and the JVM or streams are left in a bad state.
   Ideally I want to reuse JVMs for efficiency to avoid startup delays.

ERROR com.cp.matlab.servlet.MatlabExecutor.doPost(MatlabExecutor.java:141) - java.util.concurrent.ExecutionException: java.io.StreamCorruptedException: invalid type code: E0
java.util.concurrent.ExecutionException: java.io.StreamCorruptedException: invalid type code: E0
	at net.viktorc.pp4j.impl.JavaProcessPoolExecutor$JavaSubmission.getResult(JavaProcessPoolExecutor.java:452) ~[pp4j-2.2.jar:?]
	at net.viktorc.pp4j.impl.JavaProcessPoolExecutor$JavaSubmission.getResult(JavaProcessPoolExecutor.java:409) ~[pp4j-2.2.jar:?]
	at net.viktorc.pp4j.impl.ProcessPoolExecutor$InternalSubmission.getResult(ProcessPoolExecutor.java:471) ~[pp4j-2.2.jar:?]
	at net.viktorc.pp4j.impl.ProcessPoolExecutor$InternalSubmissionFuture.get(ProcessPoolExecutor.java:569) ~[pp4j-2.2.jar:?]
	at net.viktorc.pp4j.impl.JavaProcessPoolExecutor$CastFuture.get(JavaProcessPoolExecutor.java:502) ~[pp4j-2.2.jar:?]
	at com.cp.matlab.servlet.MatlabExecutor.doPost(MatlabExecutor.java:130) [classes/:?]
	at com.cp.matlab.servlet.CalcExecutorTest$1.run(CalcExecutorTest.java:65) [test-classes/:?]

2. If I use "jvmPool.submit(new Calc(json),true);" then it works, but not all submissions are completed. I typically get 15 of 16 completed, but if i send 24 submissions i also get 15 completed. The rest dont show errors, but are never processed. Something in the submission queue maybe?

Its possible there is a better way to do this, I'm open to suggestions.

I am running my application on a jar, and the classPath (jar:file:/xx.jar!/BOOT-INF/classes!/) is generating a java.nio.file.FileSystemNotFoundException

Hi,

is it possible to include modules / module paths in created java processes?

I'm trying to spawn JavaFx Applications in their own processes to encapsulate them, but have a small GUI just to give a visual indicator of theri progress. But since JavaFX ist now separated, it needs to be added as modules... Without that, the ClassLoader can't find the JavaFX Application-class...

See here for an example: https://blog.idrsolutions.com/using-javafx-with-java-11/

When I create a simple JavaProcessPool new JavaProcessPoolExecutor(new JavaProcessManagerFactory<>(new SimpleJavaProcessConfig()), ... ), the classpath of the calling application will be in the child process as well, but this isn't true for modules.

I have outputted the module path in the calling app an in the child app (System.getProperty("jdk.module.path")) - for the calling, it contains javaFx, for the child it's just null.

It would be great, if the modules from the calling app could be added to the child process / to the child process's module path (–module-path ... and –add-modules ...). Or is there any workaround to get this happening?

Any thoughts? :-)

For me, one reason to use separate processes, is to be able to terminate them, for example if they are running too long or aren't responding anymore. The child-processe's code isn't always behaving wrt. thread interrupts, so I need to have the big gun and kill the process.

In PP4J it seems to be possible in AbstractProcessExecutor, but for the JavaProcessPool, I don't see any chance to get there... Cancelling the returned Future doesn't have any effect on the process.