Demo of time comparison with contracts enabled and disabled

In this repository we compare the time performance of a project compiled with contracts enabled and disabled. Examples are compiled with EiffelStudio 19.5

SCOOP disabled. Void checking in runtime disabled.

Iterations: 200000000

Runs of the same program: 10

All times are in seconds

All examples are compiled with -finalize option.

• no_keep compiles the program without the -keep option. All other examples are compiled with -keep.
• all_enabled has contracts enabled for both the target and the cluster
• only_target has contracts enabled for the target, and disabled for the cluster
• only_cluster has contracts enabled for the cluster, and disabled for the target
• all_disabled has contracts disabled for both the target and the cluster

We see that removing contracts from the code gives use the benchmark time of 0.6 seconds on average.

The all_enabled example and the only_cluster examples effectively evaluate the contracts. This gives us the time of the program with contracts at around 6.25 seconds (the average).

The only_target example effectively disables the contract checking for the root class (enabled for the target but disables for the actual code). Together with the all_disabled example this gives us the time of around 2.7 (the average) seconds to run the program with contracts disabled.

So, compiling with contracts increases the time from 0.6 to 2.7 seconds => 4.5 times increase.

Enabling the contracts increases the time from 2.7 to 6.25 seconds => further 2.31 times increase.

The finalized C code is also present in this repository (subdirectories of results: no_keep, all_enabled etc.). This is to simplify the browsing experience for anyone interested 😄

The produced C code differs only in the egc_project_version value. All the code is otherwise the same.

Indeed, enabling contractss for the whole target and then disabling them for all clusters (the only cluster in this example), effectively disables contracts for the whole program.

Here again the only difference is the egc_project_version.

Enabling contracts for both the whole cluster and once again for all its clusters (the only cluster in this example) should have the same effect.

Discarding assertions by omitting -keep during compilation will result in all assertions-related code to be skipped during compilation to C.

The easiest to observe is the {APPLICATION}.work feature. We will look at the diff of the C3/ap124.c file which contains the code for APPLICATION:

diff -u <(sed -n '73,89p' results/no_keep/C3/ap124.c) <(sed -n '100,155p' results/all_disabled/C3/ap124.c)
 void F280_2080 (EIF_REFERENCE Current)
 {
 	GTCX
+	RTEX;
+	EIF_INTEGER_32 ti4_1;
+	EIF_INTEGER_32 ti4_2;
+	RTSN;
+	RTDA;
 	RTLD;
 	
 	RTLI(1);
 	RTLR(0,Current);
 	RTLIU(1);
 	
+	RTEAA("work", 279, Current, 0, 0, 2802);
+	RTSA(Dtype(Current));
+	RTSC;
 	RTGC;
+	RTIV(Current, RTAL);
+	if ((RTAL & CK_REQUIRE) || RTAC) {
+		RTHOOK(1);
+		RTCT("x.abs <= 200", EX_PRE);
+		ti4_1 = *(EIF_INTEGER_32 *)(Current+ _LNGOFF_0_0_0_0_);
+		ti4_2 = eif_abs_int32 (ti4_1);
+		RTTE((EIF_BOOLEAN) (ti4_2 <= ((EIF_INTEGER_32) 200L)), label_1);
+		RTCK;
+		RTJB;
+label_1:
+		RTCF;
+	}
+body:;
+	RTHOOK(2);
 	(*(EIF_INTEGER_32 *)(Current+ _LNGOFF_0_0_0_0_))++;
+	RTHOOK(3);
 	if ((EIF_BOOLEAN) (*(EIF_INTEGER_32 *)(Current+ _LNGOFF_0_0_0_0_) > ((EIF_INTEGER_32) 200L))) {
+		RTHOOK(4);
 		*(EIF_INTEGER_32 *)(Current+ _LNGOFF_0_0_0_0_) = (EIF_INTEGER_32) ((EIF_INTEGER_32) -200L);
 	}
+	if (RTAL & CK_ENSURE) {
+		RTHOOK(5);
+		RTCT("x.abs <= 200", EX_POST);
+		ti4_1 = *(EIF_INTEGER_32 *)(Current+ _LNGOFF_0_0_0_0_);
+		ti4_2 = eif_abs_int32 (ti4_1);
+		if ((EIF_BOOLEAN) (ti4_2 <= ((EIF_INTEGER_32) 200L))) {
+			RTCK;
+		} else {
+			RTCF;
+		}
+	}
+	RTVI(Current, RTAL);
+	RTRS;
+	RTHOOK(6);
 	RTLE;
+	RTEE;
 }

We can see that going from no_keep to all_disabled mostly introduces the contract checking code. The precondition is checked in the beginning in if ((RTAL & CK_REQUIRE) || RTAC) { ... }, the postcondition is checked in the end if (RTAL & CK_ENSURE) { ... }.

Even though we disable the contracts, the contracts code is added. Additionally, many other features and classes are compiled. TODO: explain why

These two examples differ only in two files.

The first file is E1/eoption.c.

The only difference is the line 288:

diff -u results/all_disabled/E1/eoption.c results/only_cluster/E1/eoption.c
 struct eif_opt egc_foption_init[] = {
 ...
-{0, 0, 0, {OPT_ALL, 0, NULL}},
+{63, 0, 0, {OPT_ALL, 0, NULL}},
 ...
 }

The array egc_foption_init is discussed later, you can skip directly to the next part.

This section is here for completeness sake. You may skip reading it.

The second differing file is again C3/ap124.c. While the files differ, we don't see any difference in symantics.

For example, the only difference in the work feature is using a temporary value to save the result of an expression (only_cluster) compared to using the whole expression in full:

diff -u <(sed -n '100,155p' results/all_disabled/C3/ap124.c) <(sed -n '102,158p' results/only_cluster/C3/ap124.c)
-	if ((EIF_BOOLEAN) (*(EIF_INTEGER_32 *)(Current+ _LNGOFF_0_0_0_0_) > ((EIF_INTEGER_32) 200L))) {
+	ti4_1 = *(EIF_INTEGER_32 *)(Current+ _LNGOFF_0_0_0_0_);
+	if ((EIF_BOOLEAN) (ti4_1 > ((EIF_INTEGER_32) 200L))) {

Similar differences occur in the make feature:

diff -u <(sed -n '60,91p' results/all_disabled/C3/ap124.c) <(sed -n '60,93p' results/only_cluster/C3/ap124.c)
-	loc1 = RTLNS(eif_new_type(784, 0x01).id, 784, _OBJSIZ_0_0_0_1_0_0_0_1_);
-	(nstcall = -1, F785_5095(RTCW(loc1)));
+	tr1 = RTLNS(eif_new_type(784, 0x01).id, 784, _OBJSIZ_0_0_0_1_0_0_0_1_);
+	(nstcall = -1, F785_5095(RTCW(tr1)));
+	loc1 = (EIF_REFERENCE) tr1;

...

-	loc2 = RTLNS(eif_new_type(784, 0x01).id, 784, _OBJSIZ_0_0_0_1_0_0_0_1_);
-	(nstcall = -1, F785_5095(RTCW(loc2)));
+	tr1 = RTLNS(eif_new_type(784, 0x01).id, 784, _OBJSIZ_0_0_0_1_0_0_0_1_);
+	(nstcall = -1, F785_5095(RTCW(tr1)));
+	loc2 = (EIF_REFERENCE) tr1;

The last difference is in the begining of the make feature:

diff -u <(sed -n '46,48p' results/all_disabled/C3/ap124.c) <(sed -n '46,48p' results/only_cluster/C3/ap124.c)
-	RTLR(1,Current);
-	RTLR(2,loc2);
-	RTLR(3,tr1);
+	RTLR(1,tr1);
+	RTLR(2,Current);
+	RTLR(3,loc2);

We should consult the definition of the RTLR macro in eif_macros.h from Eiffel Software's Runtime:

/* Macros used for local variable management:
...
 *  RTLR(x,y) register `y' on the stack at position `x'
...
 */
 
#ifdef EIF_VOLATILE
/* We do a cast to avoid a possible C compiler warnings as `y' will be of type `EIF_REFERENCE EIF_VOLATILE'. */
#define RTLR(x,y) l[x] = (EIF_REFERENCE *) &y
#else
#define RTLR(x,y) l[x] = &y
#endif

Where l seems to be a pointer to the stack.

The macro EIF_VOLATILE can be redefined to be either an empty define (expands to an empty string) or to the volatile modifier. This seems to be needed for exception handling because EIF_VOLATILE is redefined to be volatile for functions having a rescue clause (std_byte_code.e):

if rescue_clause /= Void then
    buf.put_new_line_only
    buf.put_string ("#undef EIF_VOLATILE")
    buf.put_new_line_only
    buf.put_string ("#define EIF_VOLATILE volatile")
end

So, the difference is only in the order of these local variables appearing on the stack.

Overall, we see that the program code is semantically the same. The only real difference is in the egc_foption_init array as was shown before.

Read further for the discussion of that difference.

TODO: continue here