How to Use the Perlmutter Template about funcx HOT 4 CLOSED

Hello ax3l,

I see the confusion. Those configurations are not processed by any Jinja filter. Those fields are meant to spell out where the human needs to modify the YAML configuration in their own setup.

HOWEVER ... we are nearing the release of a multi-user endpoint set of code that does what you have described. While the work is oriented around an administrative install, it will also be available to non-administrative users and would enable the workflow you've described: a templated approach to running multiple endpoints.

We made our development efforts public at ParslFest 2023 back in October. The talk (~7m) and slides are available; look for "Globus Compute Multi-User Endpoint".

from funcx.

Hi @ax3l! Public news? No, not yet, unfortunately. We're still working out a couple of kinks. We're aiming to have it ready for prime-time in about a month.

On the other hand, if you just can't wait and would like to be an early adopter, then let me point you to the current tl;dnr instructions:

• MULTIUSER_SETUP.md

These instructions are geared toward an administrative install, but the MEP also works when installed or run as a normal (non-root or otherwise non-power) user. Consequently, you may have to translate some of the instructions and/or concepts. (For example, as a non-power user, identity mapping will not work.)

We're still working on proper documentation yet (c.f. ~a month). Meanwhile, we're happy to have real users give it a whirl — there's no accounting for real users fleshing out what you didn't think about while developing!

from funcx.

Hi Kevin,

Oh that is cool, can't wait for the release of that feature! Nice talk!

from funcx.

Hi @khk-globus, I am curious if there are news on the release that ships templated approach to running multiple endpoints? In particular, I would love if I could define an endpoint on an HPC machine only once and scale the number of nodes with a runtime parameter per job/task.

from funcx.