homerreid / libgdsii Goto Github PK

OS X doesn't have the /proc pseudo-filesystem, so running GDSIIConvert logs:

12/15/19::12:38:01: Mem[0] before,during,after={0,0,0},delta=0
12/15/19::12:38:01: Mem[1] before,during,after={0,0,0},delta=0
12/15/19::12:38:01: Mem[2] before,during,after={0,0,0},delta=0
12/15/19::12:38:01: Mem[3] before,during,after={0,0,0},delta=0
12/15/19::12:38:01: Mem[4] before,during,after={0,0,0},delta=0
12/15/19::12:38:01: Mem[5] before,during,after={0,0,0},delta=0
12/15/19::12:38:01: Mem[6] before,during,after={0,0,0},delta=0

I assume Windows has a similar result. It would be better to log that the data wasn't available (absence of data != actual data value of zero). GetMemoryUsage() is prototyped to return an unsigned int, but hardcoded to return 0 and that value is ignored. If it returned a value that signifies success vs failure, then the caller (main()) can call ::Log with a different string to indicate this situation.

I'm using meep to import polygons from gds files. As far as I understand, meep is using libGDSII to perform this action.

It would be useful if libGDSII could support importing (layer, datatype) combinations (in stead of just the layer) such that meep could support this natively as well. What do you think?

See also the meep issue on this.

Would it be possible to create a 0.2 release so I can create a conda package on conda-forge? Also, can you create the release tarball using make dist so the user won't have to run autogen.sh? Thanks!

Copying the bulk of an e-mail I sent 10 days ago into public issues on this repo :)

I tried to fuzz the library through its CLI front-end, to make a quick assessment of its robustness, and I've found lots of issues. Data corruption sometimes happens in the real world.
I've fuzzed several dozens of code bases over the years, all of them being written in memory-unsafe C and/or C++, and unsurprisingly, all but one fell quickly to zzuf, afl and/or honggfuzz. For libGDSII, configuring + building with CC/CXX=afl-clang-fast and starting a fuzzing job took less than 15 minutes.

You'll find the offending files in the "output" folder from the attached tarball:
libgdsii_fuzz_afl_20190817.tar.gz

As hinted above, the results, are... not pretty:

1. the time to first crash, with an abysmal file corpus (single 0-byte file !), is around a second;
2. adding the bend-flux.gds file from the libGDSII repo produced several dozen "unique" crashes (multiple "unique" crashes can have the same final stack trace, but they went through different paths at some point, IIUC) in less than a minute;
3. adding a simple dictionary containing the codes listed at http://www.idea2ic.com/PlayWithICEDIT/Look_Inside_GDSII.html further raises the initial discovery rate for crashes.
   There are multiple paths causing nullptr dereferences, so while the AddressSanitizer instrumentation (AFL_USE_ASAN=1 make ...) did find OOB accesses worth fixing, OOB accesses which wouldn't always crash in a normal build are not the leading cause of crashes.

Some of the crashers are less than 10 bytes large, e.g. the following hex dumps:

00 00 04 00
00 04 17 01

Nothing special about the afl-fuzz invocation:

$ .../afl-fuzz -i input -o output -m none -M 1 -e gds -- .../GDSIIConvert @@ --analyze

The -e option only works with AFL++'s afl-fuzz. I could also have attempted to patch out the extension check.

The first fuzzing round finds enough issues, quickly enough, to warrant a second one, possibly using honggfuzz instead, for a change. Also, maybe more modes of GDSIIConvert, to exercise more code paths.
If you have, say, several dozen to several hundred valid small GDSII files, ranging from several bytes to ideally less than 1 KB, which use the various GDS-II codes - effectively, something like a test suite for a library like libGDSII - that would help :)

In Windows environment, I successfully compiled liGDSII according to the installation instructions with MSYS2. But when I ran the examples at command-line, I ran into the following problems. I'd like to know how to solve this problem.

C:\Pro\libGDSII\share\libGDSII\examples\GSiP>GDSIIConvert --GMSH GSiP_4_RingFilter.gds
error: unexpected end of file (aborting)

C:\Pro\libGDSII\share\libGDSII\examples\GSiP>GDSIIConvert --raw GSiP_4_RingFilter.gds
Record 0:       HEADER ( 1)  = 600
Record 1:       BGNLIB ( 12)  = 2017 3 16 23 29 47 2017 3 16 23 29 47
Record 2:      LIBNAME ( 1)  = SiEPIC-GSiP
Record 3:        UNITS ( 2)  = 0.001 1e-009
Record 4:       BGNSTR ( 12)  = 2017 3 16 23 29 47 2017 3 16 23 29 47
Record 5:      STRNAME ( 1)  = $$$CONTEXT_INFO$$$
Record 6:         SREF =
Record 7:        SNAME ( 1)  = CIRCLE
Record 8:           XY ( 2)  = 0 0
Record 9:     PROPATTR ( 1)  = 7
Record 10:    PROPVALUE ( 1)  = PCELL=CIRCLE
Record 11:     PROPATTR ( 1)  = 6
error: unexpected end of file (aborting)
error: unexpected end of file (aborting)

C:\Pro\libGDSII\share\libGDSII\examples\GSiP>GDSIIConvert -h
error: no argument given for option -h (aborting)
Usage: GDSIIConvert File.GDS [options]
Options:

 ** Output formats: **
   --raw              raw dump of file data records
   --analyze          detailed listing of hierarchical structure
   --GMSH             Export GMSH geometry to FileBase.geo (text strings to FileBase.pp)
   --scuff-rf         Write .port file defining RF ports for scuff-RF (implies --gmsh)

 ** Other flags: **
   --MetalLayer     12  define layer 12 as a metal layer (may be specified multiple times)
   --LengthUnit     xx  set output length unit in meters (default = 1e-6)
   --FileBase       xx  set base name for output files
   --verbose            produce more output
   --SeparateLayers     write separate output files for objects on each layer

Compiling libGDSII-0.21 on OS X 10.13 with Apple's compiler toolchain...

libtool: compile:  g++ -DHAVE_CONFIG_H -I. -I.. -g -O2 -MT ReadGDSIIFile.lo -MD -MP -MF .deps/ReadGDSIIFile.Tpo -c ReadGDSIIFile.cc  -fno-common -DPIC -o .libs/ReadGDSIIFile.o
ReadGDSIIFile.cc:654:50: warning: adding 'libGDSII::BYTE' (aka 'unsigned char')
      to a string does not append to the string [-Wstring-plus-int]
       *ErrMsg = new string("unknown data type " + DType);
                            ~~~~~~~~~~~~~~~~~~~~~^~~~~~~
ReadGDSIIFile.cc:654:50: note: use array indexing to silence this warning
       *ErrMsg = new string("unknown data type " + DType);
                                                 ^
                            &                    [      ]
1 warning generated.

Should it be some sort of ostringstream (like around line 579) or other type of concatenation or sprintf?

While fuzzing the library, as described in #4, I noticed the usage of /tmp/GDSIIConvert.log. This isn't good, for two reasons:

• it's a "temporary file of predictable name", in a shared directory which is world-writable in some setups. This is usually considered a vulnerability, for reasons that the Internet explains better than I do;
• GDSIIConvert writes enough data there to fill gigabytes in several hours, filling up my HDD... not good, I simply put a rm; sleep invocation in a loop to fix that, but that's ugly :)

how can i install it on windows? It says "autoreconf: command not found" when i run sh autogen.sh

and could not run make command as well

After installing libGDSII following the instructions provided in the documentation:

> git clone http://github.com/HomerReid/libGDSII
> cd libGDSII
> sh autogen.sh
> make install && sudo make install

The test case GDSIIConvert --GMSH GSiP_4_RingFilter.gds, also from the documentation, produced the following error:

GDSIIConvert: error while loading shared libraries: libGDSII.so.0: cannot open shared object file: No such file or directory

The file libGDSII.so.0 exists in /usr/local/lib and points to libGDSII.so.0.0.0 which is a 995Kb shared library verified using the nm command.

Could this be a possible bug in the autoconf script related to the runtime linker?