A number of places in opensgx, arguments are used without proper quoting, meaning paths with spaces would be treated incorrectly. For example:

measure $1 > $MEASURE

should be

measure "$1" > "$MEASURE"

When compiling the various sgx binaries and libraries (in particular inside qemu), a lot of compiler warnings are given for the sgx .c files. It would be great if these could be fixed or suppress so that it is easier to identify actual problems.

Is there a way to invoke code in an enclave programmatically, say through a c function, and get the result back?

./configure-arch works fine
make -j $(nproc) gives the following error message:
make[1]: *** No rule to make target 'target-i386/crypto_helper.o', needed by 'qemu-x86_64'. Stop.
make[1]: *** Waiting for unfinished jobs....
CC x86_64-linux-user/target-i386/seg_helper.o
Makefile:163: recipe for target 'subdir-x86_64-linux-user' failed
make: *** [subdir-x86_64-linux-user] Error 2

The function OpenSGX reeds a protocol, because otherwise, it is use as an int returning function, and truncates the higher bits of the returned address.

John

diff --git a/user/include/sgx-user.h b/user/include/sgx-user.h
index 7033ec1..f6f75c6 100644
--- a/user/include/sgx-user.h
+++ b/user/include/sgx-user.h
@@ -33,3 +33,5 @@ extern void exception_handler(void);

extern void sgx_enter(tcs_t _tcs, void (_aep)());
extern void sgx_resume(tcs_t _tcs, void (_aep)());
+
+extern void *OpenSGX_loader(char *binary, int size, long offset, int n_of_pages);

RDTSC instruction is legal in SGX enclave starting SGX2 processors that are not yet available. There is a way to support this instruction of opensgx?

Please delete

qemu/roms/seabios/config.mak
qemu/roms/vgabios/config.mak
user/sgx-test-runtime

They are derived files and cause conflicts.

John

The buffer allocated for snprintf in dump_sigstruct in sgx-utils.c is not big enough to hold the entire data structure. Q2 gets truncated by 204 bytes.

non-enclave directory has a pointer to test/simple-attest

When building qemu with make after ./configure-arch, the following error happens.

Unescaped left brace in regex is illegal here in regex; marked by <-- HERE in m/^\@strong{ <-- HERE (.*)}$/ at /home/lukenels/src/opensgx/qemu/scripts/texi2pod.pl line 320.
Makefile:463: recipe for target 'qemu.1' failed
make: *** [qemu.1] Error 255

Here is some random system information if it helps:

$ perl --version
This is perl 5, version 26, subversion 0 (v5.26.0) built for x86_64-linux-gnu-thread-multi
(with 56 registered patches, see perl -V for more detail)
.....

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 17.10
Release:	17.10
Codename:	artful

$ uname -a
Linux kaby 4.13.0-16-lowlatency #19-Ubuntu SMP PREEMPT Wed Oct 11 19:51:52 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Let me know if you want any more.

$ opensgx -s hello.sgx --key sign.key
sgx-tool: failed to locate conf/device.key: No such file or directory
$ echo $?
0

opensgx should propagate the error-status of tools that it calls so that it can be used correctly with e.g. Make.

The sgx-host code crashes when I try to run it after compilation
When run it gives an error - Illegal Instruction ( core dumped)
After putting a couple of prints I found the crash was in
static
void encls_qemu_init(uint64_t startPage, uint64_t endPage)
So I assume the sgx_init API crashes .

Hello,

Is there multithreading support available in OpenSGX?
Where can I find a minimal example of including other libraries, say <pthread.h> into a SGX program?

I just follow the instruction on the tutorial of the intra attestation, and the target enclave said "Intra attestation Fail". While signing the target enclave, it will not generate the .conf file with the "use" option, therefore, I just did "./opensgx -s path_to_sgx --key sign.key". Am I doing something wrong?

It appears that libsgx's malloc implementation does not work correctly with many small allocations. This can be verified with the following program, which incorrectly segfaults:

#include <stdlib.h>

int enclave_main(int argc, char *argv[])
{
    for(int i=0; i<38368; i++)
    {
        malloc(1);
    }

    return 0;
}

Increasing the size that is malloced to 1024 bytes prevents the segfault (although after 1359 allocations, it gives a different error, which I suspect is due to a hardcoded maximum memory limit).

I believe this bug is caused by the way libsgx handles the EAUG/EACCEPT process: when the enclave needs more memory, it will allocate a SECINFO structure (64 bytes) that is used to request another page to be mapped into its memory. However, if there are less than 64 bytes of memory available due to previous small allocations (or just bad luck), this allocation will fail and require allocation of another page, causing a recursion that overflows the stack.

This should be fixable by allocating the SECINFO on initialization and immediately after every EAUG/EACCEPT and then storing it for the next time it is needed, rather than allocating it on-demand just before the EAUG/EACCEPT.

While reading your NDSS paper on OpenSGX, a question arised regarding syscall emulation. I e-mailed with Prerit (first author in lieu of a specified corresponding author) and he suggested I bring this up here.

In §IV-B "System call emulation" you write that the enclave context is saved. In §IV-D "Trampoline and stub" you write that the enclave is exited using EEXIT and then entered using ERESUME. I looked at your libsgx source code and it does not seem that any state is saved before calling EEXIT, nor does EEXIT save state. Furthermore, EEXIT does not increase TCS.CSSA, while ERESUME decreases TCS.CSSA. If I interpret the SGX reference correctly, ERESUME is only to be called after an asynchronous exit.

Prerit pointed me at the following code: https://github.com/sslab-gatech/opensgx/blob/407d176/qemu/target-i386/sgx_helper.c#L2041 but I don't think it matches the SGX programming reference. You mention multiple times in the paper that OpenSGX is instruction-compatible. Can you elaborate on this?

I am trying to write a simple application which uses SHA256 from the OpenSSL library.

My code is stored inside user/demo/openssl and I am trying to build it with ./opensgx -c user/demo/openssl/hellosgx.c. I am getting the following errors:

devel/sgx/opensgx/user/demo/openssl/hellosgx.c:47: undefined reference to SHA256_Init devel/sgx/opensgx/user/demo/openssl/hellosgx.c:48: undefined reference to SHA256_Update' devel/sgx/opensgx/user/demo/openssl/hellosgx.c:49: undefined reference to SHA256_Final

Does anyone know what the problem might be?

Edit: sorry for the spam. There was a mistake in the Makefile.

I noticed that some preliminary work has been done on making PolarSSL's (now named mbed TLS by the way) net_* functions work. For example, the code in net.c has been modified to use sgx_memcpy and sgx_memset. Are there any plans to extend this to support establishing TLS connections inside the enclave?

The modifications are likely to be relatively mechanical (e.g., changing connect to sgx_connect, bind to sgx_bind, etc.), but there will probably a lot of them. PolarSSL is modular enough that it may be possible to simply override handlers all over the place, but it would be very useful if OpenSGX provided this out-of-the-box!

How can we run programs with greater memory requirements in opensgx? I changed the NUM_EPC and ENCLAVE_SIZE in qemu/target-i386/SGX.h and rebuilt qemu, but when I run the hello program, I see no changes in pre-allocated EPC SSA region and Heap region. I was able to compile my program using opensgx but I get a seg fault when running it. I suspect it is because of the limits in the size of the enclave.

I was able to test SGX on real hardware. There are some differences in how instructions work compared to OpenSGX.

• EEXTEND requires a pointer to SECS in RBX (this is the same pointer as PAGEINFO.SECS in EADD). This is undocumented behaviour. EEXTEND generates an exception without this.
• SIGSTRUCT, various issues:
  • The byte order of HEADER, HEADER2, MODULUS, SIGNATURE, Q1, and Q2 needs to be reversed compared to what OpenSGX does currently
  • SIGNATURE is generated over a concatenation of the fields marked 'Y' in table 2-19. The fields marked 'N' are not hashed at all, OpenSGX currently hashes these as zeroed-out fields, which is incorrect.
  • SIGNATURE must use SHA256 in the RSA signature, OpenSGX uses SHA1. This requirement is undocumented.
  • ENCLAVEHASH is just a normal SHA256 hash over the input blocks. I'm not sure what OpenSGX is currently doing, but it is not correct. Just concatenating the input of every sgx-crypto.c:sha256update call and generating a SHA256 hash using standard methods works.

I want to use opensgx to develop an application that can encrypt messages. I’m wondering how i can compile OpenSSL and use it in an enclave. Can you please add this to your tutorial?
Besides,i think it will be better if the tutorial includes usages of all available user-level APIs.
Thank you!

The GDB debug plug-ins in opensgx/gdb/ didn't work for me because the indenting was off. A quick look in the source code of the git repo showed me that tabs and spaces are used together.

I've fixed the problem, if wanted I can make a pull request.

Would be great if you could pick a license for this and place it in ./LICENSE.

sgx-tool requires the device key to be placed in a subdirectory called conf/. This also does not seem to be overrideable by a command-line parameter. Given that this is, in practice, a require parameter, it should be a required argument, and sgx-tool should refuse to run if it is not given.

In function load_rsa_keys, file sgx-util.c, if fopen failed, the resulted null pointer value (fp) would still be used (after an error message shown) in the following code.

Hi, I am a frashman to study the opensgx. Could you give a detailed example to configure data structures? I am still confused about the procedure. Thanks very much!

I have tried running the provided remote-attestation example, but it doesn't proceed to success; in fact,
it seems that all three enclave programs "hang" after the call to malloc in the sgx-remote-attestation library.

Removing the call to malloc and replacing the dynamic arrays with static ones leads to progress. However, the attestation fails providing the error "Mac mismatch" .

When I tried to use printf to see what the values are of the calculated and provided MACs, garbage is output to the terminal. I am not entirely sure where things are going wrong, to be honest. Do you guys have any information on this issue/am I perhaps doing something wrong?

In case it matters I am running openSGX in kubuntu and compiled it as per the tutorial file.

Hi, when doing compiler optimization with -O2 or -O3, gcc wrongly optimizes enclu. Especially calls to EENTER, ERESUME, EEXIT might get corrupted. This is because gcc does not know about semantics of SGX instructions, which change control flow. Hence, gcc does not save caller's registers, which might get reused in the new context. --> Data corruption

The solution I used is to unify all enclu calls in a single function and mark this function noinline and returns_twice in the header file; example shown below.

sgx-user.h:
extern void enclu(enclu_cmd_t leaf, uword_t rbx, uword_t rcx, uword_t rdx, out_regs_t* out_regs) __attribute__((noinline, returns_twice));

Hello,I meet with some difficulties about Enclave's entry and exit because I can't find the related interfaces.
For example,how can I switch to my enclave program to run when the main program is running?And how to leave the enclave to carry on the main program?
Can you give me some guidance about the related interfaces?
Thanks a lot!

Hello,
I Am trying to build opensgx as mentioned here https://github.com/sslab-gatech/opensgx/blob/master/Opensgx_tutorial_v1

after clone project and run "make –j $(nproc)" it gives me "make: *** No rule to make target –j'. Stop." also when try to run "make –C libsgx" it gives "make: *** No rule to make target –C'. Stop." , "make –C user" ==> "make: *** No rule to make target `–C'. Stop."

Compile sgx library
$ make -C libsgx

I got
In file included from src/malloc/malloc.c:13:0:
src/malloc/malloc.c: In function ‘_malloc_init’:
./include/sgx-lib.h:32:5: error: inconsistent operand constraints in an ‘asm’
asm volatile("movl %0, %%eax\n\t"
^
is it because I'm compiling this project on an 32 bit operating system but running a 64 bit CPU?

It would be really useful to have proper documentation for OpenSGX. The user manual/tutorial is a good start, but it'd be great if it were in a more digestible format such as ReadTheDocs, or simply as regular text-based files. I don't see a particularly good reason for serving it as a PDF?

I'm working on building a package for opensgx for Arch Linux (i.e. a PKGBUILD), but am running into a bunch of problems with hard-coded relative paths. For example, sgx.h has a hard-coded path to ../../qemu/target-i386/sgx.. Naturally, I can patch past all of these, but it would be great if the code base was less tightly coupled so that, say, sgx.h could be placed in /usr/include.

I can't get this system compile following the instructions in README.md
Should I adjust something according to my OS and CPU version?
well, the
$./configure-arch
$make works fine
but when I get into the ../user/polarssl_sgx dir
I got this when make:

aes.c: In function ‘sgx_aes_setkey_enc’:
aes.c:506:9: error: ‘aes_padlock_ace’ undeclared (first use in this function)
if( aes_padlock_ace == -1 )
^
aes.c:506:9: note: each undeclared identifier is reported only once for each function it appears in
aes.c: In function ‘sgx_aes_setkey_dec’:
aes.c:605:9: error: ‘aes_padlock_ace’ undeclared (first use in this function)
if( aes_padlock_ace == -1 )

would you please help me trouble shooting this?
Thanks a lot!

On Ubuntu 14.04 LTS after a pull this morning.

When I run "make -C user", I get the error message:

cc -g -Iinclude -Ishare/include -Wall -pedantic -Wno-unused-function -std=gnu11 -fno-stack-protector -fvisibility=hidden -c -o sgx-loader.o sgx-loader.c
sgx-loader.c:36:18: fatal error: gelf.h: No such file or directory
#include <gelf.h>
^
compilation terminated.

When I went to install libelf, I was told it is obsolete.

John

Hi,

I'm having problems using pipes for enclave/non-enclave communication. It seems that the system calls called by the main_enclave don't work in a correct way. For example the mkdir doesn't create a directory but a file and the fifo file is not generated by the mknod. What am I doing wrong? I should execute differently the enclave program? Sorry but I just started with opensgx. Thanks

I am debugging an opensgx application which uses openssl library within an enclave.
It works fine, but a call to ssl_get_algorithm2(s) crashes the app with:

CPU_SIGNAL_HANDLER 11
user-exec.c Called exception_action
Exception Action
Debug Raise Exception: RBP: 5029ad78   RSP: 5029aa50 EIP: 50002f77
POE_PAGE
MAPERR
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
./../sgx: line 11: 88827 Segmentation fault      $QEMU "$@"

which might be somehow related to #30

to debug further i wanted to recompile the openssl library after I included debug prints in the ssl_get_algorithm2 function.

when i try to link the newly produced (libssl.a, libcrypto.a) library files to my opensgx program it crashes straight away with:

CPU_SIGNAL_HANDLER 11
user-exec.c Called exception_action
Exception Action
Debug Raise Exception: RBP: 4000800460   RSP: 4000800460 EIP: 4042a4
POE_PAGE
MAPERR
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
./../sgx: line 11: 68674 Segmentation fault      $QEMU "$@"

which looks like the same error

i have tried increasing the NUM_EPC and HEAP_PAGE_FRAMES in each header as advised in #30 with no luck

finally my question:
how were the openssl static library files in libsgx folder generated?
do I need to include some specific ./Configure options to make it compatible with qemu?

so far i've simply tried

./configure
make

and change in the openssl Makefile:
PROCESSOR=386

grateful for any pointers,
nnm77

The error constants at https://github.com/sslab-gatech/opensgx/blob/95a6d31/qemu/target-i386/sgx.h#L134-L159 are incorrectly marked as hexadecimal while they should be decimal.

When I test user/test/simple-hello as following:
$ cd user
$ ./test.sh -i test/simple-hello
The result is:
root@ubuntu:~/opensgx/user# ./test.sh -i test/simple-hello
make: “test/simple-hello”is newest。
kern in count : 2
kern out count : 2

encls count : 9471
ecreate count : 1
eadd count : 557
eextend count : 8912
einit count : 1
eaug count : 0

enclu count : 0
eenter count : 0
eresume count : 0
eexit count : 0
egetkey count : 0
ereport count : 0
eaccept count : 0

mode switch count : 0
tlb flush count : 0

TCS address : 4fffd000
Pre-allocated EPC SSA region : 0x2000
Pre-allocated EPC Heap region : 0x12c000
Later-Augmented EPC Heap region : 0x0
Total EPC Heap region : 0x12c000
hello sgx!

number of executed instructions on CPU #0 = 123488558

'enclu count'、'eenter count'、'eexit count'、'mode stitch count' and 'tlb fluch count' are zero.It is not same as the result in Opensgx_tutorial_v1.pdf(the last two page) shows. Now,I feel so confused and don't know the reason....

PS:Tested platform:Linux ubuntu 3.19.0-25-generic #26~14.04.1-Ubuntu SMP Fri Jul 24 21:16:20 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Ubuntu runs on on VMware Workstation

When a system call fails, it should be possible to read the error out of errno. However, this doesn't seem to propagate through the trampoline and into the enclave? This makes patterns like eager-reading very difficult:

bytes = recv(s, buf, sizeof(buf), MSG_DONTWAIT);
if (bytes < 0 && errno == EAGAIN) {
    /* no data for now, wait for any byte */
    bytes = read(s, buf, 1);
}

I was implementing secret provisioning on top of the remote attestation code provided. I have found the following issues in the code:

• You are attempting to sign a SHA-256 digest with a 128-bit RSA key in the quoting enclave. RSA cannot be used to sign data larger in size than the key. Fixing this would involve either using SHA-1 (not recommended) or using larger RSA keys.
• As per the Intel SGX spec, the signature is stored in place of the mac in the report structure. However, you declared that as a 16-byte buffer this is smaller in size than 256-bits required for the signature (assuming an RSA key of 256-bits). I fixed this by utilizing a second structure that I called quote_t because I found a few instances where the size of report_t is hard coded . This also allowed me to specify a larger signature size that can be used with an RSA-2048 key. Bear in mind that RSA-128 is trivial to break.
• The calls being made to rsa_sign and rsa_verify specify data_len = 0 which is only allowed when you define the type of the md used as input to the function. However, you are not doing so and as a result the signature is being generated over a buffer of size 0 and verified over a buffer of size 0. If you check polarSSL's source code, this returns success without carrying out any computation. I would suggest defining the type of message digest instead of using the arbitrary_data flag.
• Finally, this is more of a gripe I have with your implementation than an actual issue, however, the quoting enclave, according to Intel's spec, has a known public key. This indicates that the keys in use are not ephemeral. However, you are generating RSA keys on every execution of the attestation process. I may have missed something while reading the spec, but if not, I think that utilizing a pre-computed key will make the attestation execute faster on QEMU and also adhere more to the original spec.

N.B.: I can generate a pull request if you wish with the modifications I outlined but it will contain the provisioning code as it is intermingled in the remote attestation process.

Hi, I have some doubts on the part of remote attestation.When I add "puts(rsa_E)" or "puts(rsa_N)" to and follow the tutorial to test remote attestation part, there is no output about the rsa keys. Does it mean that the two variables are null?If not,how can I get their values?

opensgx currently uses both openssl, polarssl, and mbedtls -- is there a particular reason why you don't use a single library for all crypto operations?

sgx_accept will crash with a segmentation fault if address or address_len is NULL (which they are permitted to be), as it will unconditionally call memcpy on those arguments.

When I do sgx_read on a network socket, the return value seems to always be 0 on success, regardless of how many bytes were actually read?

I'd like to use OpenSGX to write and test real SGX applications on local machines which don't support SGX instructions - and then be able to run the same application on machines that do support SGX with minimal changes.

OpenSGX does not seem to support function calls that are available on real SGX systems (For example, I can't find any reference to sgx_create_enclave() ).

Is there any way you'd suggest I could do this? Or is OpenSGX simply not the right tool?

Thanks

I cannot compile the software in the push two days ago on Ubuntu. I have enclosed one fix. but I could not fix the many errors generated from the top-level make command.

John

diff --git a/user/polarssl/Makefile b/user/polarssl/Makefile
index ce780c5..1a99c9c 100755
--- a/user/polarssl/Makefile
+++ b/user/polarssl/Makefile
@@ -3,7 +3,7 @@ OBJS = rsa.o entropy.o ctr_drbg.o bignum.o md.o oid.o asn1parse.o sha1.o
md5.o ripemd160.o net.o aes_cmac128.o

%.o: %.c

• $(CC) -c -fPIC -Wall -o $@ $<
• $(CC) -c -I../include -fPIC -Wall -o $@ $<

all: $(OBJS)

I've now updated the Arch package so that it works with HEAD. It might be good to include installation instructions similar to those you provide for Fedora and Ubuntu? I'd submit a PR, but wasn't sure how you wanted them included given that the flow is somewhat different when a package is available:

Installation if you have an AUR helper like yaourt installed should be as simple as

yaourt -S opensgx-git

Otherwise, download, make, and install like any other AUR package:

mkdir build && cd build
curl -L -O https://aur.archlinux.org/cgit/aur.git/snapshot/opensgx-git.tar.gz
tar -xvf opensgx.tar.gz && cd opensgx
makepkg -sri

Note that the install you get with this procedure puts device.key in /etc/sgx/device.key, and places the various sgx-* binaries in /usr/bin. Furthermore, header files are installed to /usr/include/sgx/, and are stripped of their sgx- prefix, so you'll have to change hello.c to use #include <sgx/lib.h> instead of #include <sgx-lib.h>.

I once was able to run Lua within an enclave. When I try to compile the code now, I get a linkage error saying setjmp and longjmp is undefined. Here is a small program that demonstrates the problem.

$ cat user/demo/sj.c

include <setjmp.h>

if defined NOSGX

int main()
{
jmp_buf env;
if (setjmp(env)) {
return 0;
}
else {
longjmp(env, 1);
}
}

else

include <sgx-lib.h>

void enclave_main()
{
jmp_buf env;
if (setjmp(env)) {
sgx_exit(NULL);
}
else {
longjmp(env, 1);
}
}

endif

$ ./opensgx -c user/demo/sj.c
cc -c -g -Iinclude -Ishare/include -Wall -pedantic -Wno-unused-function -std=gnu11 -I../libsgx/include -I../libsgx/musl-libc/include -fno-stack-protector -static -fvisibility=hidden -o demo/sj.o demo/sj.c
cc -static -nostdlib -nostartfiles -Wl,-T,sgx.lds demo/sj.o -o demo/sj.sgx ../libsgx/sgx-entry.o ../libsgx/libsgx.a ../libsgx/libpolarssl-sgx.a ../libsgx/libc-sgx.a
demo/sj.o: In function enclave_main': /home/ramsdell/rep/opensgx/user/demo/sj.c:22: undefined reference tosetjmp'
/home/ramsdell/rep/opensgx/user/demo/sj.c:26: undefined reference to `longjmp'
collect2: error: ld returned 1 exit status
make: *** [demo/sj.sgx] Error 1
rm demo/sj.o
$