sitemule / ileastic Goto Github PK

We need functions for accessing the query string.

Something like

• contains key
• get value
• get values

I think that is a very important feature as many UI frameworks work with query string parameters f. e. for JSONP callback prefix.

It also introduces the need for having some type of list as a type because you can have the same key multiple times in the query string.

Hi,

Procedure il_getParmStr() issues a "Pointer not set for location referenced." error, when no query parameters have been specified with the request URL.

   124      PSLIST pParmList = pRequest->parmList;                  
   125                                                              
   126      for (pNode = pParmList->pHead; pNode; pNode=pNode->pNext) {
   127         PSLISTKEYVAL parm = pNode->payloadData;                   
   128         if (keyLen == parm->key.Length) { 

Line 126: Pointer not set for location referenced, because pParmList is null.

It works fine, when at least the question mark has been added at the end of the URL.
You can easily reproduce the problem with the "PLUGIN" example.
I am not sure whether or not that is a bug or works as designed. But since il_getRequestHeader() works more gentle, I would expect il_getParmStr() to behave the same.

Regards,

Thomas.

In order to handle user sessions, there needs to be a way to create cookies. This will allow developers to determine unique sessions.

• il_getCookie(request:'name') - returns string or pointer to string
• il_hasCookie(request:'name') - return bool
• il_setCookie(response:'name':'value') - void
• il_deleteCookie(response:'name') - void

There may be more to add as well, but just an idea.

Then when there is a single session cookie, we are able to create logins, etc.

As Basic Auth is a very common way for authentication we need a plugin to resolve the username and password from the Authorization HTTP header.

The credentials could be put on the storage map for the request by this we would decouple the parsing of the HTTP header from the authentication mechanism (which could be a user account, validation list, external service, ...). Another plugin for authentication could later pick up the values from the storage map and do the authentication.

If using the storage map for authentication poses a security risk (storing the password in the map) then we could add some configuration for the basic auth plugin and then resolve the authentication function from the config thus keeping it dynamic.

Hi,

The HELLOWORLD.RPGLE example program listens on port 44001, which is the same port that program STATICFILE.RPGLE listens on.
Your README.MD file states, that HELLOWORLD.RPGLE should listen on port 44000. Please fix HELLOWORLD.RPGLE to listen on port 44000.

Thank you,

Thomas.

Currently the configuration is hard coded inside the program. Meaning every change in the configuration needs a change/compile/test/deploy cycle of the program. It would be much nicer if the configuration could be saved outside of the code and loaded through some loader instance.

Ideally this would be done via some interface as this would allow loading the configuration from different sources.

Possible sources could be:

• Database file
• XML stream file
• YAML stream file

In a servlet REQUEST/requestDS, there should be a way to determine the request type:

dcl-proc myservlet;

    dcl-pi *n;
        request  likeds(REQUESTDS);
        response likeds(RESPONSEDS);
    end-pi;

    printf(request.endpoint);
    printf(request.method); //GET, POST, etc

end-proc;

Of course, if it is PUT, there needs to be a way to get content of the body. (il_getBody(request) ???)

This snippet of code runs just fine:

    pJson = json_NewObject();

    // add new elements to the object
    json_SetInt    (pJson : 'a'  : 1 );
    json_SetDec    (pJson : 'b'  : 12.34);
    json_SetDec    (pJson : 'c'  : 123);
    json_SetBool   (pJson : 'ok' : 2>1);  // true
    json_SetStr    (pJson : 's'  : 'hi');  // String

    il_responseWrite(response: json_AsText(pJson));

This does not:

    pJson = json_NewObject();

    // add new elements to the object
    json_SetInt    (pJson : 'a'  : 1 );
    json_SetDec    (pJson : 'b'  : 12.34);
    json_SetDec    (pJson : 'c'  : 123);
    json_SetBool   (pJson : 'ok' : 2>1);  // true
    json_SetStr    (pJson : 's'  : 'hi');  // String

    il_responseWriteStream(response : json_stream(pJson));

And it just comes out blank.

Any ideas?

Like express can, ILEastic should provide a way to serve static files or directories.

I was thinking something like:

• il_staticFile
• il_staticDir

For example:

dcl-proc myservlet;

    dcl-pi *n;
        request  likeds(REQUESTDS);
        response likeds(RESPONSEDS);
    end-pi;

    Select;
        When (request.endpoint = '/'); //Index
            il_staticFile('./path/to/file.html');

        When (%Subst(request.endpoint:1:6) = '/other'); // - /other/file.html
            il_staticDir(request:'./path/to/');
    Endsel;

end-proc;

This idea obviously requires routing (#1).

At the moment when the end point handler / servlet creates an escape message no message is sent to the client at all and the client waits for a response.

The client should get at least a message with the status code 500 that an error occured when an escape message is sent from the servlet code.

What happens when the servlet already sent half the payload and then creates an escape message? We already have sent a chunk with the HTTP status (probably 200).

                     Additional Message Information                      

Message ID . . . . . . : MCH3601 Severity . . . . . . . : 40
Message type . . . . . : Escape
Date sent . . . . . . : 11/27/20 Time sent . . . . . . : 11:08:24

Message . . . . : Pointer not set for location referenced.
Cause . . . . . : A pointer was used, either directly or as a basing
pointer, that has not been set to an address.

                        Display Message Details                          

Message ID . . . . . . : MCH3601 Severity . . . . . . . : 40
Date sent . . . . . . : 11/27/20 Time sent . . . . . . : 11:08:24
Message type . . . . . : Escape
From . . . . . . . . . : CECUSER CCSID . . . . . . . . : 65535

From program . . . . . . . . . : ILEASTIC
From library . . . . . . . . : ILEASTIC
From module . . . . . . . . : ILEASTIC
From procedure . . . . . . . : findRoute
From statement . . . . . . . : 10

To program . . . . . . . . . . : ILEASTIC
To library . . . . . . . . . : ILEASTIC
To module . . . . . . . . . : ILEASTIC
To procedure . . . . . . . . : findRoute
To statement . . . . . . . . : 10

Does the repository need the two submodules?

• ILEfastCGI
• noxDB

Can these be removed?

Thanks!

demo03 uses copybook JSONparser.rpgle which doesn't exist, thus will not compile.

Hi,

Whatever I do, I always get an error when attempting to install ILEastic on our IBM i.

1. Cloned the repository as described in README.md.

2. Started a SSH session, navigated to the ILEastic folder and execute the MAKE command. Error: "make: The error code from the last command is 1." after "libllist -a ILEASTIC ILEFASTCGI;".

3. Naviagted into the ILEfastCGI folder and successfully excuted the GMAKE command.

4. Went to the noxDB folder and successfully executed the GMAKE command.

5. Went back into the ILEastic folder and executed the MAKE command again. Error: "make: The error code from the last command is 126." after "liblist -a ILEASTIC ILEFASTCGI;".

Any ideas?

Regards,

Thomas.

This is a fixed timestamp in the 'Date' header in the response.

https://github.com/NielsLiisberg/ILEastic/blob/master/src/ileastic.c#L95

This should be fixed ASAP.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Date

Currently it is not possible in a plugin to identify which endpoint/route is serving the request. It would be nice to be able to add an identifier to the il_addRoute function and add that to the request data structure so that the plugin can access it and act dynamically to that value.

Use case: Have a plugin that adds CORS header corresponding to the endpoints used based on a provided configuration.

To run microservices - FastCGI would be a nice options is run from i.e. nginx. This however requires that the job is not in multithreaded mode but serialized

It would be nice to be able to register a request filter at the service. Every request before routing it to the endpoint would need to pass through the filter. The filter would decide if the request is valid or not.

This would also enable us to implement Basic Authentication.

Good morning,
I´m pretty excited about your project, but i was wondering that GET, PUT,POST, and DELETE are the only supported HTTP-methods. Would it be possible to support the other methods defined in RFC7231 and RFC5789 in a future release?

il_addRoute(config : %paddr(getProperty) : IL_GET : '^\/api\/mapping\/[^\/]+/{property}$');

with the following URL: http://host:44000/api/mapping/16/value/dummy

results in a parameter binding for property of "value/dummy"

In this case as the path parameter should be the last segment of the URL is should have resulted in not finding any route as this route does not match the URL.

Request: http://ibmi:44001/

il_getParmStr(request : 'client');

results in escape message.

MCH3601: Pointer not set for location referenced.

From program . . . . . . . . . :   REQUEST        
  From library . . . . . . . . :     ILEASTIC     
  From module  . . . . . . . . :     API          
  From procedure . . . . . . . :     il_getParmStr
  From statement . . . . . . . :     3            

It would be nice if the datachunks example shows the actual chunking.

Everytime a route is not found there is an escape message in the job log.

MCH3601 Pointer not set ...

Von Programm . . . . . . . . . :   ILEASTIC            
  Von Bibliothek . . . . . . . :     ILEASTICUP        
  Von Modul  . . . . . . . . . :     ILEASTIC          
  Von Prozedur . . . . . . . . :     cleanupTransaction
  Von Anweisung  . . . . . . . :     2                 

Hi

ILEastic is awesome, but there needs to be a way to handle the routing for requests.

For example, I'd like one 'servlet' to be able to handle multiple routes. Right now ILEastic cannot handle routes.

My initial thought is that the endpoint/url of the request could be passed in with the REQUEST/requestDS struct.

For example:

dcl-proc myservlet;

    dcl-pi *n;
        request  likeds(REQUESTDS);
        response likeds(RESPONSEDS);
    end-pi;

    Select;
        When (request.endpoint = '/'); //Index
            il_write(response:'<p>index</p>');
        When (request.endpoint = '/other'); //Index
            OtherHandler(request:response);
    Endsel;

end-proc;

What do you think?

Many thanks,
Liam Barry

We need to be able to hook into the response before it is send over the network.

Sometimes the response needs to be modified before sending f. e. CORS support, #34 . CORS needs to add some extra headers to the response.

Currently there is only a message which tells you that the thread could not be started when a request comes in. I think it would be better to test on startup if the current job has the parameter "allow multithreading = *yes". This job attribute could be retrieved with the system API QUSRJOBI, format JOBI0400, "Allow multiple threads".

The last HTTP header entry is not added to the headerlist.

ileastic.c , function parseHeaders:

pRequest->headers.Length is without the ending 0d 0a

    begin =  pRequest->headers.String;
    headend = begin + pRequest->headers.Length; 
    while (begin < headend) {
        for (;*begin == 0x20; begin ++); // Skip blank(s)
        end  = memmem(begin, headend - begin , eol , 2); // Find end of line   <--- won't find the last end because of "missing" eol at end of headers

(please forgive me if I break any protocols for reporting issues...this is my first time contributing to a project in github)

Procedure jwt_isExpired in module source /ileastic/plugins/jwt/jwt.rpgle has 2 issues:

1. The parameter pPayload is not being parsed to extract the exp value. Instead, variable payload is parsed. I do not see any pointer references or any eval statements to move pPayload into payload.

2. When the payload variable is parsed into the json variable, and the exp value is extracted using the json_getInt() procedure, the results "can be" a -1. The code immediately following the json_getInt() checks of the exp value is >= 0. Since the expired variable defaults to *off (or not expired), then if the json_getInt() procedure fails to return an expiration value, the procedure will always return not expired.

These 2 issues are causing jwt_isExpired to erroneously return "not expired" for jwt's that are expired.

I know how to fix the issues but have not kept "up-to-speed" on how to use Github to change the code.

json = json_parseString(pPayload);
exp = json_getInt(json : 'exp' : -1);
if (exp >= 0);
expTimestamp = UNIX_EPOCH_START + %seconds(exp + %int(offsetSeconds));
expired = (now >= expTimestamp);
else;
expired = *on;
endif;

Many JavaScript application need CORS support for their REST services if JSONP is not used.

il_getParmStr does not work with the following HTTP message when used with a default value.

GET /?client HTTP/1.1
Host: localhost

The query parameter client has no value (which resembles an empty string). So if il_getParmStr is called like this

client = il_getParmStr(request : 'client' : defaultClient);

It should return an empty string and not the default value because the parameter does exist but it seems it is not added to the list of paramters, see https://github.com/sitemule/ILEastic/blob/master/src/ileastic.c#L255 .

Unit test request , test case test_rootResourceWithEmptyQueryParameterDefaultValue .

The BIND step fails on the CRTSRVPGM command, because parameter BNDSRVPGM lists service program JSONXML, which is not part of the repository or any other repository referenced by .gitmodules.
For now, removing JSONXML from the CRTSRVPGM command solved the problem.

I have an issue building. When I run with qsh (per default in the makefile) it fails fast with "gmake not found". If I run with bash then it does everything fine but in the end, the ILEASTIC service program is not created because command "liblist" is not found.

I solved it by changing

  BIND_LIB=$(BIN_LIB)
  # The shell we use
  # SHELL=/QOpenSys/usr/bin/qsh
  #         liblist -a $(LIBLIST);\

Out of the box, the script does not finish (in my environment at least but I tried to keep everything as default).

In many cases you just need to send a HTTP status back to the caller without any content. Currently you must do the following:

  response.status = IL_HTTP_NO_CONTENT;
  il_responseWrite(response : '');

It would be much better to have that in one call.

il_responseWriteStatus(response : IL_HTTP_NO_CONTENT);

SSL / TLS would be nice if services are run with a public interface.

It would be nice to have constants for the common HTTP status and media types (MIME types).

Currently you can just retrieve the value of the Accept header. But the header can have multiple values like

application/json,application/xml,text/plain

or even with a priority

application/json,application/xml;q=0.6,text/plain;q=0.8

So some extra procedures for handling different media types would be great.

pointer il_getAcceptedMediaTypes(request); The returned pointer could be a simple list instance which is already in the project. We would just need to expose the list API by adding them to the binder source of the ILEastic service program.

varchar il_getPreferredAcceptedMediaType(request); This would return the accept http header with the highest priority. Could return / if none was provided.

varchar il_isMediaTypeAccepted(request : 'application/json' : ... : 'application/xml'); This would return the media type which ranks the highest in the list of accepted media types. As RPG cannot do variable number of parameter we would need to limit them to a fixed number and declare them as *nopass.

It was great, if ILEastic could be changed to use a non-blocking listener socket so that it can periodically call a isShutdownRequested() callback procedure. To keep ILEastic 100% backward compatible, it could use the existing blocking socket if the callback is not set.
Background information:
First of all I think it is bad style to end jobs with ENDJOB OPTION(*IMMED).
Second, in case we make the decision to use ILEastic, it should (maybe even it must) be possible to integrate it into our framework that we use for controlling asynchronously running jobs (never ending jobs).
Of course I know, that ILEastic can not accept requests while it is in the callback procedure. But is that a blocker, if ,for example, the callback just checks a flag, a data area, a data queue or %shtdn()?

Some actions need to specify HTTP headers on the response, f. e. the creation of a resource should return the URI to the new resource in the returned HTTP message (Location header is a way to do it).

Currently escape message are handled in ileastic.c serverThread function caught by the exception handler and passed to handleServletException which returns a 500 message to the client.

It is currently not possible to somehow log the escape message. It would be great if we could have an interface to get all necessary information on a not handled escape message so that we could write individual plugins to handle this information f. e. log it somewhere.

I need a way to determine whether the request is coming from the local machine (e.g. the same system the server is running on).

Could also be used for logging.

Add native support for JSON Web Tokens.

The server routes to the wrong resource because of case insensitive path matching.

Path /time and request /Time results in a successful match which is wrong.

Unit test: routing.rpgle , test case test_caseSensitivity

Hi,

I received the following error message, when attempting to install the example programs:
liblist: 001-0081 Fehler CPF2110 gefunden beim Hinzufügen von Bibliothek NOXDB in Bibliotheksliste.

Please remove references to NOXDB from the make file of the example programs, because it seems, that NOXDB is not or no longer required. Or am I missing something? At least I could install the example programs without NOXDB.

Regards,

Thomas.

Hi,

Your example program STATICFILE.RPGLE assumes that il_serveStatic() returns *OFF on errors:

    // Serve any static files from the IFS
    // if (not il_serveStatic (response : file));   <== Wrong.
    if (il_serveStatic (response : file));   <== Should be
        response.status = 404;
        il_responseWrite(response : 'File ' + file + ' not found');
    endif;

In fact il_serveStatic() returns *ON on errors and *OFF on success.
Please fix STATICFILE.RPGLE to correctly check for errors.

Regards,

Thomas.

We need a function which returns a list (or in worst case an array) of query string values.

http://localhost/postalcodes?id=32423&id=32425&id=32429&city=Minden&country=Germany

Function suggestion:

list  il_getParmList(request : 'id');

or

array  il_getParmList(request : 'id');

At the moment you can only get the full resource path, f. e. /api/book/ean/1234567890123
In many cases I need to get a specific part of the resources.

API suggestion: il_getResourcePart(request : 4) to get the fourth part of the resource path which would return 1234567890123 in this example.

Is there a way to compile ILEastic with TGTRLS(V7R1M0)? We have 60 machines with that version requirement.

I think I had troubles with FastCGI if that is something we can remove from the compile.

Hi

In PR #6, I committed demo03.rpgle. While it is working code, it does show a bug with the request.queryString contents.

If you compile demo03.rpgle, start the program and then send a request to http://host:13337/helloworld/howareyou?hello=yes, this is the response:

method: GET
url: /helloworld/howareyou?hello=yes
resource: /helloworld/howareyou
queryString: ?hello=yes HTTP/1.1 Host: 10.1.
protocol: HTTP/1.1
content: 
contentType: 
counter : 1 
counter : 2 
counter : 3 
counter : 4 
counter : 5 
counter : 6 
counter : 7 
counter : 8 
counter : 9 
counter : 10 

As you can see, I am seeing the queryString as I'm expecting, but it looks like the protocol is appended on the end.

Is this expected or is this a bug?