earthscope / libmseed Goto Github PK

The program:

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>

#include <libmseed.h>

int main (int argc, char **argv) {

    int retcode;

    int32_t trace_number;
    int32_t segment_number;
    int64_t sample_number;

    MSTraceList *mstl = NULL;
    MSTraceID *cur_tid;
    MSTraceSeg *cur_segment;

    char seg_time[32];

    retcode = ms_readtracelist (&mstl, argv[1], 0, -1., -1., 0, 1, 0, 1);
    if(retcode != MS_NOERROR) {
        printf("error reading traces");
        printf("\t%s", ms_errorstr(retcode));
        return 1;
    }

    mstl_printtracelist (mstl, 0, 1, 0);

    // iterate of the traces
    cur_tid = mstl->traces;
    for(trace_number=0; trace_number<mstl->numtraces; ++trace_number) {

        // iterate over the segments
        cur_segment = cur_tid->first;
        for(segment_number=0; segment_number<cur_tid->numsegments; ++segment_number) {

            printf("%s\n", ms_hptime2isotimestr(cur_segment->starttime, seg_time, 1));

            // iterate over the samples
            for(sample_number=0; sample_number<cur_segment->numsamples; ++sample_number) {
                printf("%ld, ", sample_number);
            }

            cur_segment = cur_segment->next;
        }

        cur_tid = cur_tid->next;
    }

    return 1;

} /* End of main() */

produces

   Source                Start sample             End sample        Hz  Samples
IU_COLA_00_LHZ    2010,058,06:50:00.069539 2010,058,07:59:59.069538 1   4200
Total: 1 trace(s) with 1 segment(s)
2010-02-27T06:50:00.069539

When compiled against branch master (c23cc3d).

This is because mstl->traces->first->datasamples is NULL.

when run against ./example/test.mseed. I was expecting I'd see 4200 numbers rapidly scroll down my screen. Am I doing something wrong?

Hi,
I noticed the minor version number for the release 2.19 has been left to 18. Is that intentional?
Thank you in advance for your time.

Cristiano.

I apologise for the gumption in my making this suggestion, but I wonder given that develop now has a couple of bug fixes that v3.0.9 doesn't (i.e., #65 and #70 plus the crc32c warning issue), whether it might be nice to tag a v3.0.10?

Of course you will have your own processes and timelines which I understand, but it will save us carrying a patch in https://github.com/JuliaBinaryWrappers/libmseed_jll.jl and also have libmseed cross compiled and working for all these targets.

using make gcc runs with error: make: *** No rule to make target gcc. Stop.
but i need lmplatform.h for my files but simply using make does not make it.
so how can I fix this problem?
regards

hello
i'm reference your code to pack and unpack the miniseed with steim2, but i found some confuse about the page146 figure 14 steim 2 compression data format on in SEED Format Version 2.4 .
int the figure,30bit,15bit,10bit,8bit,6bit,5bit,4bit all show the d0 to d.n(MSB to LSB), but in your code, the 8bit is difference(in little-endian), the it is dn to d0.

     /* 4 x 8-bit differences */
      else if (diffcount >= 4 &&
               bitwidth[0] <= 8 && bitwidth[1] <= 8 &&
               bitwidth[2] <= 8 && bitwidth[3] <= 8)
      {
        if (encodedebug)
          ms_log (1, "  W%02d: 01=4x8b  %d  %d  %d  %d\n",
                  widx, diffs[0], diffs[1], diffs[2], diffs[3]);

        word = (union dword *)&frameptr[widx];

        word->d8[0] = diffs[0];
        word->d8[1] = diffs[1];
        word->d8[2] = diffs[2];
        word->d8[3] = diffs[3];

        /* 2-bit nibble is 0b01, only need to set 2nd bit */
        frameptr[0] |= 0x1ul << (30 - 2 * widx);

        packedsamples = 4;
      }

In this part of the code, diffs [0] is the lowest byte in U32 if it is in small-end format. Other formats store diffs [0] from the high to the low of U32. Normally, d0, d1, d2 are from the high to the low of U32 or from the low to the high?
The following figure shows debug data for this source code compression.

This is a screenshot of the IRIS SEED Format Version 2.4 August, 2012。

When/if the new miniSEED 3 specification is approved:

1. Verify library matches specification
2. Add tests specifically for mseed3

Requested changes to avoid CRAN check compilation warnings when submitting package that uses libmseed. The use of SCNd64 generates these warnings on their Windows builder:
warning: ISO C does not support the 'I64' ms_scanf length modifier [-Wformat=]

fileutils.c, line 471
/(replacing) sscanf (hdrstr, " %" SCNd64, &datasize);/
datasize = scan_d64(hdrstr,0,&endptr);

genutils.c, line 1220
/*(replacing)fields = sscanf (readline, "#@ %" SCNd64, &expires); */
fields = 0;
expires = scan_d64(readline,2,&endptr);
if (endptr - readline + 2 > 0) fields++;

genutils.c, line 1244
/*(replacing) fields = sscanf (readline, "%" SCNd64 " %d ", &leapsecond, &TAIdelta); */
fields = 0;
leapsecond = scan_d64(readline,0,&endptr);
if (endptr - readline > 0) fields++;
fields += sscanf(endptr, " %d ", &TAIdelta);

libmseed.h, inserted after line 811
extern int64_t scan_d64(char *str, int offset, char **endptr);

doc/ms_readmsr.3:

-On the sucessful read and parsing of a record \fBms_readmsr\fP and
+On the successful read and parsing of a record \fBms_readmsr\fP and
 \fBms_readmsr_r\fP return MS_NOERROR and populate the MSRecord struct
 at \fI*ppmsr\fP.  Upon reaching the end of the input file these
 routines return MS_ENDOFFILE.  On error these routines return a
 libmseed error code (defined in libmseed.h)

-On the sucessful read and parsing of a file \fBms_readtraces\fP and
+On the successful read and parsing of a file \fBms_readtraces\fP and

doc/msr_unpack.3

-On the sucessful parsing of a record \fBmsr_unpack\fP returns
+On the successful parsing of a record \fBmsr_unpack\fP returns

Hi developers,
usually SIMD is used for fast data storing/retrieval, and there is no exception for miniSEED.

As a result, is there any planned SIMD support for fast data storing/retrieval?

The license that originally came with the "pivotal" 64-bit time routine, incorporated into libmseed as ms_gmtime_r(), contains this clause:

Redistribution and use in binary form must reproduce an
acknowledgment: 'With software provided by http://2038bug.com/' in
the documentation and/or other materials provided with the
distribution, and wherever such acknowledgments are usually
accessible in Your program.

To comply as much as possible, add the acknowledgement to the README and possibly to the man pages for the time routines.

The CGPM recently announced their recommendation that no new leap seconds be added between years 2035 through 2135:
https://www.nature.com/articles/d41586-022-03783-5

While there is chance the ITU may not adopt this recommendation, and a push by some members to avoid new leap seconds sooner, the tide does seem to be turning against new leap seconds. There is also no guarantee that new leaps will not be added before 2035.

It would be highly advantageous for the list of leap seconds to be embedded in the library and not need to rely on an external file or other source.

This idea is to store the current leap seconds list statically, and allow updates from an external source in case new leap seconds are added.

A byte range decoration on a file name, parsed as <file>@startbyte-endbyte, does not produce a proper HTTP Range header when the start is a value of 0. To illustrate, this request:

http://website.tld/data/file.mseed@0-4096

Produces the following Range header:

Range: bytes=-4096

Which incorrectly omits the starting 0, and worse, is then a valid Range value for data from the end of the specified object, potentially fetching incorrect range.

If a MS2 blockette's text field is mapped to an extra header and contains non-UTF-8 characters, parsing will fail with messages like:

ERROR: mseh_add_event_detection(): Cannot set: Detector
ERROR: msr3_unpack_mseed2(XFDSN:GT_BDFB__B_H_Z): Problem mapping Blockette 201 to extra headers
ERROR: Cannot read TEST-B201-bad-detector-name.mseed: No miniSEED data detected

Test data is attached:
TEST-B201-bad-detector-name.mseed.zip

A segfault can occur in msr_unpack_data if the samplecnt field of the MSRecord is negative because memcpy is called with a negative third argument.

Pull request #19 contains a test and a fix that should resolve this issue

The seisan2mseed package uses lmp_ftello, but it is not exported from the shared library or DLL. This would be fine if you use the builtin copy, but for packaging we prefer to remove that and link with a common copy. Since that common copy doesn't export lmp_ftello, seisan2mseed cannot be built against it.

A need for caller-specified memory management through specified malloc(), realloc(), and free() functions has been identified. Need internal hooks for setting and using specified functions, otherwise the normal defaults.

Files that have records with different publication versions and are being read with mstl3_readbuffer_selection() will result in traces that are ordered front to back. This is not a deal breaker but I feel this should not be the case.

This file (https://www.dropbox.com/s/sibg35lgb4t7t0x/test.ms3?dl=1) contains three records from stations AA, BB, and CC. publications versions go from 1 to 3.

The resulting trace list is in the correct order if the publication versions are identical but as soon as they differ the order is reversed.

The libmseed 2.x branch is still (as of 4 June 2021) licensed under the LGPL v3.0.

Most users of libmseed are still on version 2.x. Some of them strongly prefer the Apache License v2.0 to the LGPL v3.0.

Can you please re-license the libmseed 2.x branch under the Apache License v2.0, then, can you please release the next version of libmseed v2.x straight after?

Thank you very much for your efforts!

Hello @chad-iris,

I started working on the Debian packaging for libmseed, mostly because I'd like to have mseed2sac and sac2mseed packaged. I encountered a few licensing issue you may want to fix:

A portion of genutils.c comes with a peculiar license:

https://github.com/iris-edu/libmseed/blob/master/genutils.c#L1584

This license:

• Is probably non-free, as it requires a portion of code (the char array) to remain untouched. (I know this is borderline, but it's enough for the package to be rejected from Debian.)
• It is not compatible with the GPL, as it contains an additional advertising clause. See https://www.gnu.org/licenses/license-list.en.html#OriginalBSD and https://www.gnu.org/licenses/bsd.html for some background. This means that the software is probably not distributable anyway.
• The fact that the file has multiple licenses makes downstream packaging difficult, but this is a secondary issue.

Will you consider replacing pivotal_gmtime_r() with an alternative with a better license? As an alternative, do you think it's feasible to reimplement the function and properly release it under the GPLv3?

Thank you!

The HTPTIME2EPOCH() and EPOCH2HPTIME() macros do not encapsulate the provided expression, this means a call such as:

MS_EPOCH2HPTIME (a+b)

does not work as expected since the conversion is applied only to b.

The expression is easily encapsulated with something like:

#define MS_EPOCH2HPTIME(X) (X) * (hptime_t) HPTMODULUS

but perhaps the entire thing should also be encapsulated as well.

Before committing this change, we need to check the internal usage to make sure any casting is not being done that relies on the current usage.

I think this line:

https://github.com/iris-edu/libmseed/blob/master/libmseed.h#L22

should go now that libmseed is Apache licensed.

I have built libmseed v3.0.2 successfully using make shared.

make install fails with the following message:

cp: cannot stat 'doc/libmseed-UsersGuide': No such file or directory

In fact doc/libmseed-UsersGuide is missing from the source tree and there is no rule in the Makefile to create it.

The install target also contains this command, which will fail because the files doc/ms*.3 do not exist:

@cp -a doc/ms*.3 $(DESTDIR)$(MAN3DIR)/

A segfault occurs in ms_readleapsecondfile if it is called twice, because in the second call leapsecondlist is non-null, lastls is null, and dereferencing it as lastls->next = ls dereferences a null pointer.

Pull request #17 should resolve this issue.

Hi Chad,
thanks a lot for making and maintaining this package. I deeply appreciate your work.

Last week I stumbled over a very surprising problem in a specific combination of gcc version and compiler flags. It is affecting (at least, not sure if only) reading of float64 mseed files. The problem manifests when the compiler aggressively optimizes ms_gswap8a().

To reproduce the problem, try

• gcc 11.2.0 (arch linux ships this as the default since about 2 weeks)
• libmseed 2.19.6
• CFLAGS="-O3 -flto" make
• make test should now show FAILED Test pack-Float64-encoded.test, Compare pack-Float64-encoded.test.ref pack-Float64-encoded.test.out

The problem does NOT occur when I do either of the following:

• downgrade the compiler, or
• add -fno-strict-aliasing, or
• prevent gcc from optimizing ms_gswap8a by embedding it in

  #pragma GCC push_options
  #pragma GCC optimize ("O0")
  void ms_gswap8a(...) {...}
  #pragma GCC pop_options

• use -O1 or remove -flto, or
• let the code call ms_gswap8 instead of ms_gswap8a.

I am not sure what is the right way to fix the problem, what do you think?

Best regards
Sebastian

Hi Chad,
I get an unresolved symbol when building msrepack.c in example folder:
msrepack.o: In function main': msrepack.c:(.text+0x16d): undefined reference to unpackencodingformat'

in libmseed.h, I see you declare an extern int unpackencodingformat;
defined in unpack.c
Perhaps I was a bit too restrictive with the exported external symbols?

regards,

Pierre

In order to make the generated HTML documentation fully functional offline please include a local copy of the IRIS logo instead of fetching it at runtime, here:

https://github.com/iris-edu/libmseed/blob/master/doc/footer.html#L4

I think that's the only line to change. I understand the documentation is meant to be pushed to GitHub pages, however a copy of the generated HTML files is going to be included as a Debian/Ubuntu package for offline reference. Think of the people doing field work in Antarctica :-)

Hi Chad,

Currently I am using your awesome work.
When I try to call ms3_readtracelist_timewin() I always receive an error about 'No miniSEED data detected'.
However, when I use ms3_readselectionsfile() to read the selection then call ms3_readtracelist_selection() to select the data within the time window, this problem disappears.
After some investigation, I find there is a difference when creating MS3Selections struct and filling its data between ms3_readtracelist_timewin() and ms3_readselectionsfile():

1. In ms3_readtracelist_timewin(), the function DOES NOT set the value of pubversion.

  selection.sidpattern[0] = '*';
  selection.sidpattern[1] = '\0';
  selection.timewindows   = &selecttime;
  selection.next          = NULL;

2. Whereas in ms3_readselectionsfile() the function DOES set the value of pubversion to zero if there is no pubversion specify in the input file.

    /* Test for "SourceID  [Starttime  [Endtime  [Pubversion]]]" */
    if (fieldidx == 1 ||
        (fieldidx == 2 && isstart2) ||
        (fieldidx == 3 && isstart2 && isend3) ||
        (fieldidx == 4 && isstart2 && isend3 && ms_isinteger(fields[3])))
    {
      /* Convert publication version to integer */
      pubversion = 0;
      if (fields[3]) /* It Pubversion fields exists) */
      {
       ...
      }

IMHO, I think we can set the pubversion to zero in ms3_readtracelist_timewin().
If you think this is a great idea, I will send a PR at once :)

When i run "nmake -f makefile.win" under windows 10 using watcom,got the following errors:

extraheaders.c(392): Error! E1063: Missing operand
extraheaders.c(392): Error! E1009: Expecting ')' but found '{'
extraheaders.c(392): Note! N2002: 'memcmp' defined in: F:\WATCOM\H\string.h(145)
extraheaders.c(392): Error! E1151: Parameter count does not agree with previous definition
extraheaders.c(392): Error! E1009: Expecting ')' but found '0'
extraheaders.c(392): Error! E1063: Missing operand
extraheaders.c(392): Warning! W111: Meaningless use of an expression
extraheaders.c(392): Warning! W111: Meaningless use of an expression
extraheaders.c(392): Error! E1009: Expecting ';' but found ','
extraheaders.c(392): Warning! W111: Meaningless use of an expression
extraheaders.c(392): Warning! W111: Meaningless use of an expression
extraheaders.c(392): Warning! W111: Meaningless use of an expression
extraheaders.c(392): Warning! W111: Meaningless use of an expression
extraheaders.c(392): Error! E1009: Expecting ';' but found '}'
extraheaders.c(392): Warning! W107: Missing return value for function 'mseh_add_event_detection'
extraheaders.c(392): Error! E1061: Expecting data or function declaration, but found ','
extraheaders.c(392): Error! E1026: Invalid declarator
extraheaders.c(392): Error! E1009: Expecting ';' but found '6'
extraheaders.c(392): Error! E1061: Expecting data or function declaration, but found 'constant'
extraheaders.c(392): Error! E1026: Invalid declarator
extraheaders.c(392): Error! E1009: Expecting ';' but found ')'
extraheaders.c(392): Error! E1061: Expecting data or function declaration, but found ')'
extraheaders.c(392): Error! E1026: Invalid declarator
extraheaders.c(392): Error! E1009: Expecting ';' but found ')'
extraheaders.c(392): Error! E1061: Expecting data or function declaration, but found ')'
extraheaders.c(393): Error! E1026: Invalid declarator
extraheaders.c(393): Error! E1009: Expecting ';' but found '{'
extraheaders.c(393): Error! E1061: Expecting data or function declaration, but found '{'
extraheaders.c(394): Error! E1147: Too many errors: compilation aborted
Error(E42): Last command making (extraheaders.obj) returned a bad status
Error(E02): Make execution terminated

And everything is fine when i make the libmseed 2.x

The msr_parse()/msr_parse_selection() routines use an 'int' to specify the 'recbuf' length, this effectively limits the buffer to ~2GB size.

Change record buffer length to int64_t to support larger buffer sizes.

This has not been reported as a problem, the ticket is a placeholder for consideration at major version change.

Background: Currently the "selection" capability of the library limits the file reading functionality to miniSEED records that contain data for the selection. In most selections with a specified time window, this results in a bit more data, before and after, than is specified in the selection.

An enhancement could provide the capability to further trim the traces such that the data is limited to exactly what is specified in the selection.

One idea for this is to provide a new function that takes a trace group or list along with a selection and performs the trimming. A caller would first call one of the file reading routines with a selection and then call the trimming function with the same selection.

Hello,

I'm just starting to use the libmseed library. Following the examples, I generated a miniSEED file using the lm_pack example in the examples folder.

If I parse it using the lm_parse example, I'm able to read the generated miniSEED file. If I try to do the same using Obspy, I get a "Unknown format for file test.mseed" error.

I'm unsure whether that C script is generating a faulty miniSEED file, or not generating a proper miniSEED file at all?

Custom prefixes longer than MAX_LOG_MSG_LENGTH should result in error messages being logged. They are not logged because the variable argument list in two calls to ms_log_l does not contain a format string.
Pull request #23 should resolve this issue.

The read-recordlist test is failing with v3.0.7 (the first version of v3.x I try to compile). Full transcript of the build/test session:

$ git checkout v3.0.7
Note: switching to 'v3.0.7'.
[...]
HEAD is now at 67f0bee Add latest version to ChangeLog

$ git status
HEAD detached at v3.0.7
nothing to commit, working tree clean

$ make
cc   -c fileutils.c -o fileutils.o
cc   -c genutils.c -o genutils.o
cc   -c gswap.c -o gswap.o
cc   -c lmplatform.c -o lmplatform.o
cc   -c lookup.c -o lookup.o
cc   -c parson.c -o parson.o
cc   -c msrutils.c -o msrutils.o
cc   -c extraheaders.c -o extraheaders.o
cc   -c pack.c -o pack.o
cc   -c packdata.c -o packdata.o
cc   -c tracelist.c -o tracelist.o
cc   -c gmtime64.c -o gmtime64.o
cc   -c crc32c.c -o crc32c.o
cc   -c parseutils.c -o parseutils.o
cc   -c unpack.c -o unpack.o
cc   -c unpackdata.c -o unpackdata.o
cc   -c selection.c -o selection.o
cc   -c logging.c -o logging.o
Building static library libmseed.a
rm -f -f libmseed.a
ar -crs libmseed.a fileutils.o genutils.o gswap.o lmplatform.o lookup.o parson.o msrutils.o extraheaders.o pack.o packdata.o tracelist.o gmtime64.o crc32c.o parseutils.o unpack.o unpackdata.o selection.o logging.o

$ make test
make[1]: Entering directory '/tmp/libmseed/test'
PASSED Building lm_pack.c
PASSED Building lm_parse.c
PASSED Building lm_read_buffer.c
PASSED Building lm_read_recordlist.c
PASSED Building lm_read_selection.c
PASSED Building lm_sids.c
PASSED Building lm_timestr.c
PASSED Test pack-Float32-encoded.test
PASSED Test pack-Float64-encoded.test
PASSED Test pack-Int16-encoded.test
PASSED Test pack-Int32-encoded.test
PASSED Test pack-Steim1-encoded.test
PASSED Test pack-Steim2-encoded.test
PASSED Test pack-text-encoded.test
PASSED Test parse-sids.test
PASSED Test parse-timestr.test
PASSED Test read-CDSN-encoded.test
PASSED Test read-DWWSSN-encoded.test
PASSED Test read-Float32-encoded.test
PASSED Test read-Float64-encoded.test
PASSED Test read-GEOSCOPE163-encoded.test
PASSED Test read-Int16-encoded.test
PASSED Test read-Int32-1024byte-encoded.test
PASSED Test read-Int32-128byte-encoded.test
PASSED Test read-Int32-2048byte-encoded.test
PASSED Test read-Int32-256byte-encoded.test
PASSED Test read-Int32-4096byte-encoded.test
PASSED Test read-Int32-512byte-encoded.test
PASSED Test read-Int32-8192byte-encoded.test
PASSED Test read-SRO-encoded.test
PASSED Test read-Steim1-bigendian.test
PASSED Test read-Steim1-littleendian.test
PASSED Test read-Steim2-bigendian.test
PASSED Test read-Steim2-littleendian.test
PASSED Test read-buffer-tracelist.test
PASSED Test read-detection-record.test
PASSED Test read-invalid-blockette-offsets.test
PASSED Test read-mixed-order-mixed-length-trace.test
PASSED Test read-no-blockette1000.test
FAILED Test read-recordlist.test, Compare read-recordlist.test.ref read-recordlist.test.out
PASSED Test read-selection.test
PASSED Test read-text-encoded.test
PASSED Test read-unapplied-timecorrection.test
43 tests conducted
make[1]: Leaving directory '/tmp/libmseed/test'

$ diff -u test/read-recordlist.test.ref test/read-recordlist.test.out
--- test/read-recordlist.test.ref	2019-12-03 10:42:15.734223365 +0100
+++ test/read-recordlist.test.out	2019-12-03 11:36:35.860073174 +0100
@@ -1,19 +1,19 @@
 TraceID for XFDSN:XX_TEST_00_L_H_Z (1), segments: 1
   Segment 2010-02-27T06:50:00.069539000 - 2010-02-27T07:55:51.069539000, samples: 3952, sample rate: 1
   Record list:
-    RECORD: bufferptr: 0x0, fileptr: 0x0, filename: data/Int32-oneseries-mixedlengths-mixedorder.mseed, fileoffset: 0
+    RECORD: bufferptr: (nil), fileptr: (nil), filename: data/Int32-oneseries-mixedlengths-mixedorder.mseed, fileoffset: 0
     Start: 2010-02-27T06:50:00.069539000Z, End: 2010-02-27T06:50:15.069539000Z
-    RECORD: bufferptr: 0x0, fileptr: 0x0, filename: data/Int32-oneseries-mixedlengths-mixedorder.mseed, fileoffset: 13952
+    RECORD: bufferptr: (nil), fileptr: (nil), filename: data/Int32-oneseries-mixedlengths-mixedorder.mseed, fileoffset: 13952
     Start: 2010-02-27T06:50:16.069539000Z, End: 2010-02-27T06:51:03.069539000Z
-    RECORD: bufferptr: 0x0, fileptr: 0x0, filename: data/Int32-oneseries-mixedlengths-mixedorder.mseed, fileoffset: 9344
+    RECORD: bufferptr: (nil), fileptr: (nil), filename: data/Int32-oneseries-mixedlengths-mixedorder.mseed, fileoffset: 9344
     Start: 2010-02-27T06:51:04.069539000Z, End: 2010-02-27T06:52:55.069539000Z
-    RECORD: bufferptr: 0x0, fileptr: 0x0, filename: data/Int32-oneseries-mixedlengths-mixedorder.mseed, fileoffset: 128
+    RECORD: bufferptr: (nil), fileptr: (nil), filename: data/Int32-oneseries-mixedlengths-mixedorder.mseed, fileoffset: 128
     Start: 2010-02-27T06:52:56.069539000Z, End: 2010-02-27T06:56:55.069539000Z
-    RECORD: bufferptr: 0x0, fileptr: 0x0, filename: data/Int32-oneseries-mixedlengths-mixedorder.mseed, fileoffset: 14208
+    RECORD: bufferptr: (nil), fileptr: (nil), filename: data/Int32-oneseries-mixedlengths-mixedorder.mseed, fileoffset: 14208
     Start: 2010-02-27T06:56:56.069539000Z, End: 2010-02-27T07:05:11.069539000Z
-    RECORD: bufferptr: 0x0, fileptr: 0x0, filename: data/Int32-oneseries-mixedlengths-mixedorder.mseed, fileoffset: 9856
+    RECORD: bufferptr: (nil), fileptr: (nil), filename: data/Int32-oneseries-mixedlengths-mixedorder.mseed, fileoffset: 9856
     Start: 2010-02-27T07:05:12.069539000Z, End: 2010-02-27T07:21:59.069539000Z
-    RECORD: bufferptr: 0x0, fileptr: 0x0, filename: data/Int32-oneseries-mixedlengths-mixedorder.mseed, fileoffset: 1152
+    RECORD: bufferptr: (nil), fileptr: (nil), filename: data/Int32-oneseries-mixedlengths-mixedorder.mseed, fileoffset: 1152
     Start: 2010-02-27T07:22:00.069539000Z, End: 2010-02-27T07:55:51.069539000Z
 DATA (3952 samples) of type 'i':
    -231946     -228438     -223155     -221231     -225429     -230129

With a MS3TraceList with segments of both waveform encoding == 'i' and ASCII log files encoding == 'a'
I would like to mstl3_write() to a file with 'a' as ASCII and 'i' as STEIM2, However ...write() and ...pack() require and encoding and will not write the other.

Is there a simple way to do this without traversing the segments myself?
Or a different approach altogether?

It would be very nice if the selection file format supported "Year-Month-Day" dates in addition to "Year-DayOfYear" form. The challenge is made more complex by a) the date-times are allowed to be short, specifying only a date, only an hour, etc. and b) the fields in the file can be in different columns depending on file format.

Allowing both date formats might require more strictness in the forms. Review is required to determine if this can be detected unambiguously.

Some of the bit shifting is done in a manner left undefined by C99, which leave it implementation-dependent. As far as I can tell all test systems I've tried do it the same way, implying that a negative effect is not widespread.

The undefined shifts are reported by the compiler upon request:
CFLAGS="-Wall -g -fsanitize=undefined,address"

Reported by @gillian-iris

The defines in lmplatform.h needed by libmseed.h should be incorporated directly into libmseed.h. This encapsulates all definitions and declarations needed to use the libraries in a single header.

In a test case with a record that has been repeated ad nauseam, e.g. 150,000 times, the search/match algorithm in mstl3_addmsr() (aka mstl3_addmsr_recordptr()) is reduced to
its least efficient, linear search of the record list and reading becomes quite slow.

Tracking through valgrind I get an uninitialized variable issue. The order of calls looks like:

(1) ms3_readtracelist calls
(2) ms3_readtracelist_selection where line 675 in fileutils.c has int64_t fpos which calls
(3) ms3_readmsr_selection then at line 304 I get the uninitialized error which seems sensible.

Setting line 675 to int64_t fpos = 0;

This seems a little suspect since fpos might not be appropriate given it may require some consistency with the open file.

Currently the version 3 of libmseed implements a new data format also known under the name "next generation format" or "NGF" [1], following discussions within FDSN Working Group 2, and more recently also under the name "extended miniSEED" or "xSEED" by IRIS [2]. While "NGF" was chosen as a temporary name to avoid confusion with miniSEED, "xSEED" seems like a brilliant name for the new format, as it nicely symbolizes an extension of the SEED family of data formats.

Naming the new data format (NGF, xSEED, miniSEED3, ...) consistently as "xSEED" offers several advantages:

• Ensure coherency in the naming of the various "SEED" family members.

• Avoid the ambiguous use of the name "mseed" or "miniSEED" for two very different formats and supporting libraries.

• Have two separate libraries libmseed and libxseed.

The current use of the name "libmseed" for a library supporting current miniSEED in version 2 and xSEED/NGF in version 3 is not only confusing but also problematic from a technical point of view. Continuation of the version 2 currently takes place somewhat hidden within a branch of libmseed. Whereas the main branch (version 3) libmseed is not compatible with most software that once adopted libmseed in good faith. Using the same name for two different libraries will also make linking against both at least cumbersome if not impossible, whereas separate libmseed and libxseed would allow to more cleanly distinguish between the two libraries and link against both wherever needed.

[1] FDSN/miniSEED3-TechnicalEvaluation#26
[2] https://iris-edu.github.io/xseed-specification

Current output in verbose printing of record contents contains the bit order in least-significant bit first (LSB-first):

                  flags: [00100000] 8 bits
                         [Bit 2] Clock locked

For example in msr3_print().

Bits in documentation are much more commonly ordered with most-significant bit first (MSB-first), matching binary number representation.

Thanks to @crotwell for the eagle eyes to notice this.

lm_pack example generates a mseed file that apparently is correct but it can not be opened with other software as python or matlab.

The previous msr_pack() incremented the msr->starttime value for each record that it produced. The new msr3_pack() does not increment the time and thus each record has the same start time. Oops.

There are a few typographical errors in the ms_log.3 man page.
Pull request #21 fixes the typographical errors.

As reported for mseedconvert: EarthScope/mseedconvert#3, which includes test data.

The time correction applied bit in the activity flags is never set when writing format version 2. On read of format 2, the library always applies any unapplied time corrections. This should be reflected on writing of format 2 by setting the time correction applied bit flag when the correction is non-zero. Somewhere in here: https://github.com/iris-edu/libmseed/blob/master/pack.c#L993

In libmseed.h, unistd.h needs to be included on GNU/Hurd systems (__GNU__) and GNU/kFreeBSD systems (__FreeBSD_kernel__ && __GLIBC__). Relevant issue:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=890571

Current release is named 'v2.19.5.tar.gz' but once the file is downloaded, and a few weeks (or minutes depending on memory) pass, you have no idea what it is. Much better to name it
libmseed-2.19.5.tar.gz. Also nice if the untarred directory matches the filename. Currently the release untars to libmseed-2.19.5 which would match the above suggestion.

Seems like the standard is packagename-.tar.gz without a v in the tag.

Add mstl3_readbuffer_selection() counterpart to ms3_readtracelist_selection().

Hi Chad!

With the major API changes from version 2 to 3 of the library, will or can existing v2 code break if a data provider send miniseed data written with the v3 library?

This would be a severe problem for legacy, real-time and mission-critical systems that rely on libmseed.