@cdondrup Does the QTC implementation return np.arrays?

I think @yianni 's idea was that the multiple qsr implementations would return the same type so we can quickly switch between them. RCC3 implementation returns strings.

I am currently testing a new probabilistic version of arg_distance (opening a PR soon) which I want to get together with my QTC relation. I thought, since future=True returns a dict with the name of the QSR and the computed relations, that the future option would let me request multiple QSR with one service call, getting the response in a merged dict. Looking at how the QSR for the current request is chosen this only works when self.request_message.which_qsr is a single string. I think this would be easy to transform into a loop if future==True and type(self.request_message.which_qsr)==list to run the request for an array of QSRs, merge the dicts and then send one reply. This would not alter the default behaviour but enable the new one.

This would, in my case, allow to make one service call instead of two which saves a bit of computation and traffic. I could get cracking at implementing this but I wanted to ask what you have envisioned for the future option @yianni as currently it just turns a string into a dict and I have no real clue what you want to do with it.

Currently not supported which leads to -inf loglikelihoods if the state chains is not producible by the given HMM.

Arguments like validate and no_collapse have to be bool. Implement custom check to throw exception if they are not.

At the moment the bounding box returned is in the format of [bottom_left_x, bottom_left_y, top_right_x, top_right_y]. This has created confusion to me as usually bounding boxes are in vision at least represented in terms of top_left, bottom_right coordinates; and hence I am thinking of changing it as such at some point.

So any views on that?

change q=args.quantisation_factor to dynamic_args={"quantisation_factor": args.quantisation_factor}

Use dictionary namespace, see #90 for discussion.

Allow client to optionally request for which objects QSRs will be created. If it is empty then revert to default behavior. Also related to #26

We (@pet1330 and I) just had a look at the rcc3 output of the example client:

Response is:
0: o2,o1:dc; o2,o3:dc; o1,o3:dc; o1,o2:dc; o3,o2:dc; o3,o1:po; 
1: o2,o1:dc; o2,o3:dc; o1,o3:dc; o1,o2:dc; o3,o2:dc; o3,o1:po; 
2: o2,o1:dc; o2,o3:dc; o1,o3:po; o1,o2:dc; o3,o2:dc; o3,o1:po;

How can o1,o3 be disconnected and o3,o1 be partially overlapping at the same time? Also the objects shouldn't overlap looking at the coordinates and bounding boxes. Something might be wrong with the logic here.

As --future returns QSR dictionaries, the only QSR not complying is QTC as there were some things not clear to me when I try to edit it.

Either @yianni to have another look or @cdondrup

Not high priority at the moment.

Make dynamic_args of QTCS to have a namespace. Easiest resolve, to prefix them with qtcs_, qtcbs_, etc as needed.

See #25

When requesting qsrs from multiple programmes at the same time, the reset command is not thread safe and sets the request message to None. What is that good for in the first place? The values are always overridden when the function is called anyway. @yianni is this used for anything or can I just remove that all together? If so, I can fix that together with #83

I have a feeling that custom checks return is buggy due to passing by reference. Test and fix if that's the case.

@yianni I have a question about the ros client:

Is this meant as a tool that can be used like in the ros_example_client.py to have an easy way of interfacing the ros service without bothering about the topics and service type? Because if it is, which would be great, then this line has to go into the ros_example_client.py. The reason being, if you want to create a ros node that calls this you will most likely have to have a call to init_node as well. ROS sadly doesn't like two of those in a single node. If you don't mind I will fix this.

As per #90 just to consistent.

I see you already moved it. OK, I will test and see if there are any issues.

Test that

self.client_node = rospy.init_node("qsr_lib_ros_client_example")  # needed for rospy.get_rostime() in the request method

that was moved from qsrlib_ros_client.py to example_ros_client doesn't cause a problem with

req.header.stamp = rospy.get_rostime()

in qsrlib_ros_client.py/QSRlib_ROS_Client/make_ros_request_message

@cdondrup
Quick question: qtcb(o1, o2) is calculated with pose of o2 at t or t-1? Relevant bits below but pos_l[-2] not helping...

k = [input_data.trace[t0].objects[o1_name].x,
                             input_data.trace[t0].objects[o1_name].y,
                             input_data.trace[t1].objects[o1_name].x,
                             input_data.trace[t1].objects[o1_name].y]
                        l = [input_data.trace[t0].objects[o2_name].x,
                             input_data.trace[t0].objects[o2_name].y,
                             input_data.trace[t1].objects[o2_name].x,
                             input_data.trace[t1].objects[o2_name].y]
                        qtc_sequence = np.append(qtc_sequence, self._create_qtc_representation(
                            k,
                            l,
                            quantisation_factor
                        )).reshape(-1,4)

pos_k = np.array(pos_k).reshape(-1, 2)
pos_l = np.array(pos_l).reshape(-1, 2)
rot_RL = self._orthogonal_line(
            pos_k[-2],
            np.append(pos_k[-2], (pos_l[-2]-pos_k[-2]))
        ).reshape(-1,2)
        #print "rot_RL", rot_RL
        trans_RL_k = self._translate(
            [rot_RL[0], rot_RL[1]],
            (rot_RL[0]-rot_RL[1])/2
        )
trans_RL_k = self._translate(
            [rot_RL[0], rot_RL[1]],
            (rot_RL[0]-rot_RL[1])/2
        )
k = np.append(
            self._test_constraint(
                pos_k,
                trans_RL_k,
                quantisation_factor=quantisation_factor),
            self._test_constraint(
                pos_k,
                RL_ext,
                quantisation_factor=quantisation_factor,
                constraint="side")
        )

From @PDuckworth's work on trajectory activities for the G4S deployment, he is doing lots of calls with trajectories such as: (t, o1), (t, o2), (t, o3) etc. This is introducing a fair bit of overhead doing all these hundreds of calls (tens of thousands once the database fills up when learning I guess). It would be better if we could have a service that did (t, [o1, o2, o3, ...]) to reduce the overhead. Is this possible? It shouldn't be too difficult as it's simply moving the loop from one end to the other. This would be in addition to the existing calls.

This has annoyed me ever since and I don't see its use really. Can we please remove this?

@cdondrup I am thinking of adding a type field (string) to QSR class that will make life easier if a QSR object contains multiple relations within its qsr string field.

For example if you wanted to describe qualitative distance and speed between two objects you would have to have something like: QSR.qsr = "far,fast" or any other notation like +,+. This makes it difficult to remember what is what. The type could provide a hint and could also be used for parsers/decomposers if needed, e.g. in this example QSR.type = "distance,speed".

This change should not break any existing code as I will have it as an optional argument with default value being an empty string.

As described in this comment. The ability to pass QSR arguments using a user defined config file.

Showing how to use raw data and the qsr_lib to generate input for the qsr_prob_lib. Currently something like this can be found here: https://github.com/strands-project/strands_hri/blob/hydro-devel/hrsi_representation/scripts/offline_qtc_creator.py which uses a lot of stuff that is not particularly easy to read as an example.

I've noticed that QTC timestamps do not match a World_Trace object timestamps. This is expected in default behavior, i.e. validate is True and no_collapse is False (or either of the two conditions).

However, it would be nice/useful when validate is False and no_collapse is True to get back the right timestamps. Here is a function below that rectifies them:

def rectify_qtcb_timestamps(world_trace, world_qsr_trace):
        for t, tqtc in zip(world_trace.get_sorted_timestamps()[1:], world_qsr_trace.get_sorted_timestamps()):
            world_qsr_trace.trace[t] = world_qsr_trace.trace.pop(tqtc)

Can either be done class the QTC if above conditions are met, or can go in the qsrlib_utils/utils.py file.

@cdondrup What do you think?

Object_State.return_bounding_box_2d in world_trace.py is broken when width or length are nan, as it returns nan-bounding boxes and since the qsrs (RCCs in particular) have no check they return non-sense QSRs.

This is normally used in ROS and also easily loadable.
Just a suggestion.

I have succesfully implemented qsrlib's qtcb for pairwise objects. I do this by creating an instance of World_Trace() for each pair of objects in the scene. This seems inificient. Can I create one World_Trace, and pass all the object_state_series at once?

I think Yiannis said this was possible in rcc. Is it also possible in the qtc code? @cdondrup @yianni

See: http://strohel.github.io/PyBayes-doc/filters.html#particle-filter-family for example

@cdondrup out.qsr.trace has more timepoints than world.trace input.

I pass 44 poses (UUID, obj pairs), and it returns 47 traces.

See screenshot for example (highlighted in terminal).

FYI, RCC3 returns 44 timepoint (for comparison).

The new feature of computing multiple QSRs with one service call does not allow to pass individual parameters. E.g. quantisation_factor for qtcbs and mos are forced to use the same. These keys need to be uniquely renamed. In fact, all parameters belonging to a QSR should have a namespace. Easiest way to do this is to prefix with their short unique name.

I am really keen on changing the qsr field from default string to dictionary for various reasons, including better support of QSRs that consist of multiple singleton QSRs, being able to concat different QSRs etc.

Any thoughts/objections to that before I go ahead and do it?

get_sorted_timestamps in World_Trace and World_QSR_Trace do not return the timestamps sorted correctly due to them being stupid strings for easier ROS serialization.

Will submit a dirty inefficient fix for now.

This function is bugged.

The standalone example is far behind from the ros one. Rewrite so that they can be in sync with minimal effort.

Currently the quantisation_factor, validate, no_collapse, and distance_threshold are added to each object. This is unnecessary and could go into the service calls using the ini argument if this would support strings or as separate arguments.

@cdondrup Excellent initiative and would be useful I think. I have a few questions:
Can you copy and paste the README again or the message from the earlier pull request...
Can you copy and paste the link to your branch, really busy now but I would like to see the code about some things...

The two key points I would like to ask are:
a) standalone use would be really great to be honest.
b) can you ignore the string representation of the QSR and take advantage of the dict one (--future) instead (if havent done so, need to see source)... the thing is that it would be really flexible and powerful as then it allows to compute transitions and emissions on merged QSRs (eg. get RCC8, get QTCB -> merge in to one which currently is supported)

We can chat when we are both free.

Nice one!

Argh forgot to do one last change I wanted, but nvm for now... just a reminder for me.

Expose to public and make @staticmethod the QSRlib.__merge_world_qsr_traces method.

@marc-hanheide I am quite happy with the current state of the QSRlib, so if @cdondrup also agrees I would like to trigger a new release as there are significant differences and enhancements from the latest ones.

Also (probably my fault as it has my signature) for some reason if you look in the releases page it gives as "latest release" a fairly old one. Not sure how this happened (I tagged it maybe?), why or how to change it.

Whenever @marc-hanheide you (or any of the jenkins admins) have time if you could please look into these two things.

Thanks.

The QTC parts are rather unnecessarily complicated for most of the other QSRs as they implement the transitions from the CND. arg_distance should serve as an easy to recreate example for new QSRs.

Currently only qtcb and qtcc are working. qtcbc needs an additional distance value associated to the Cartesian coordinates of the two agents.

Already have written some docstrings for most of the QSRlib that I need to push.

This issue servers as a general reminder to all contributors.

It should be possible to validate qtc sequences for correctness according to the CND. This would however require to insert steps in between timestamps.
The quantisation is necessary to generate 0-states in noisy data. This quires an additional variable that can be given to the lib.

Make QTC take advantage of #33 enhancements

Would it be possible to set the default values for, e.g. x and y, to NaN or Inf instead of 0? That way it would be possible to see if the values have been entered properly. In the QTC case it could very well be that x and y are 0 but someone could also have forgotten to give them values. In order to wrie an exhaustive custom check this would have to be changed.

What title says. No rush.