espena / sosicon Goto Github PK

I'm trying to load a SOSI file to a postgis database. It works well, but I note that an empty string '' is used if a field is missing, when it is instead common to use NULL. For me it works to hack line 207 in converter_sosi2psql.cpp, to always use "NULL,", but it might be useful to others to be able to select the behavior with a command line option.

Contours have no z-data after conversion. What can be done to solve this?

I'm trying to get this dataset into PostGIS, following the instructions here.

It eventually crashes on one of the files:

Reading 0231VALGKRETSER.SOS
21213 lines parsed
Building postGIS export...
Coordinate system: ETRS89 / UTM zone 32N
std::exception

I've uploaded that file here. The problem occurs in 47 of the 435 files. Not sure if this is a bug in the data or in sosicon.

Digging further, the problem seems to be in sosicon::ConverterSosi2psql::insertLineString - where this line causes the crash since the geom string is empty. So I try this naive fix:

diff --git a/src/converter_sosi2psql.cpp b/src/converter_sosi2psql.cpp
index 747fd8e..3f3b560 100755
--- a/src/converter_sosi2psql.cpp
+++ b/src/converter_sosi2psql.cpp
@@ -400,6 +400,9 @@ insertLineString( ISosiElement* lineString,
     }

     std::string geom = ssGeomCoord.str();
+
+    if (geom.empty()) return;
+
     geom.erase( geom.size() - 1 );

     std::stringstream ss;

Now at least the conversion doesn't crash:

$ sosicon -2psql -srid 900913 0231VALGKRETSER.SOS

Sosicon, Copyright (C) 2014 Espen Andersen.
This program comes with ABSOLUTELY NO WARRANTY; for details type `sosicon -w'
This is free software, and you are welcome to redistribute it under certain
conditions; type `sosicon -c' for details.

Reading 0231VALGKRETSER.SOS
21213 lines parsed
Building postGIS export...
Coordinate system: ETRS89 / UTM zone 32N
    > Converting SOSI data to SQL...
isNumeric: 0
len: 8
isNumeric: 0
len: 4
isNumeric: 0
len: 6
isNumeric: 0
len: 18
isNumeric: 0
len: 9
isNumeric: 0
len: 30
isNumeric: 0
len: 29
isNumeric: 0
len: 19
isNumeric: 1
len: 2
    > 9 polygons processed
    > postgis_dump.sql written
    > Clean-up...
Done!

But PostGIS doesn't like the generated SQL, which includes an invalid POLYGON()):

ST_Transform(ST_GeomFromText('POLYGON())',25832)

Also, I can't convert the complete set of files without eventually hitting a segfault:

Reading 1134VALGKRETSER.SOS
8494 lines parsed
Building postGIS export...
Coordinate system: ETRS89 / UTM zone 32N
[1]    2042 segmentation fault  sosicon -2psql -srid 900913 *.{sos,SOS}

It may very well be my naive fix causing that though.

Any ideas?

I have used SOSI previously on the Norwegian Civil Aviations obstructions file and it worked fine. For some reason it does not work any more.

I am merely interested in converting the file to .shp

http://data.kartverket.no/data/temadata/nrl/NRL_Norge_UTM33_SOSI.zip

I have tried the lates command line version and the GUI which I downloaded today.

Windows 7, 64-Bit

Any ideas would be approciated

Har fått tilsendt sosi-data frå kollega som bruker Focus arealplan (som eg kjenner godt sjølv også), og skal berre lage plangrense i shape for bruk i esri, men eksporten stoppar med påstand om at programmet ikkje finn sosi-fila, sjå skjermdump.

I've downloaded SOSI files from Kartverkert and I'm trying to convert them to shp using the latest version of sosicon for Linux 64 bits.

Most of the files are encoded in ISO-8859-1, but some of them are UTF8. In this latest case if I run the tool a shape containing all of the points in the file is generated, but the extended characters in the metada are corrupted. If I reencode the file from UTF8 to ISO-8859-1 before running the tool the extended characters in metadata are OK, but the coordinates for the points are in the shape are completely wrong, they're even out of the range for numbers. No error message is displayed in either case.

I've attached a zip contaning both the SOSI files (the original, named 2019adresser_UTF8.sos and the reencoded, named 2019adresser_ISO.sos) and a corresponding folder containing the result file I get after running the tool.

Can you please have a look at them?

TIA

DemoFiles.zip

In the SOSI file converter GUI, if you include sosi-files with whitespace characters in the absolute path, these are not escaped correctly. Interestingly enough, then it does escape the output desination folder, but not the input files. This is the command which is produced in the status window:

sosicon -2shp -d "C:/Users/vikingscientist/Documents/maps/existing buildings" C:/Users/vikingscientist/Documents/maps/eksisterende bygg.sos

in which case it is producing the following error:

C:/Users/vikingscientist/Documents/maps/eksisterende not found!
bygg.sos not found!
*** Conversion completed! ***

Its not possible to either compile or run the finished binary on debian 64bit

Hi,

Thanks for this useful tool.

I am trying to import a sosi file into psql using sosicon but receiving errors like below:

ERROR:  relation "elvegsamferdsel_01.tbl0101Elveg_point" does not exist
CONTEXT:  SQL statement "ALTER TABLE elvegsamferdsel_01."tbl0101Elveg_point" ADD COLUMN "tbl0101Elveg_geom" geometry(Point, 4326)"
PL/pgSQL function addgeometrycolumn(character varying,character varying,character varying,character varying,integer,character varying,integer,boolean) line 106 at EXECUTE
SQL statement "SELECT public.AddGeometryColumn('',$1,$2,$3,$4,$5,$6,$7)"
PL/pgSQL function addgeometrycolumn(character varying,character varying,character varying,integer,character varying,integer,boolean) line 5 at SQL statement

After some digging around it seems that sosicon -2psql is generating SQL where table names are sometimes quoted and sometimes not. In particular the SELECT AddGeometryColumn single-quotes table names which seems to be translated to a double quoted table name in the ALTER TABLE statement. This causes errors for example:

This is a section of the sql file generated using sosicon -2psql showing the create table sections (emphasis is mine):

CREATE TABLE IF NOT EXISTS elvegsamferdsel_01.tbl0101Elveg_point(id_tbl0101Elveg_point INT DEFAULT nextval('elvegsamferdsel_01.tbl0101Elveg_point_serial'),datafangstdato VARCHAR(8),komm VARCHAR(4),kp VARCHAR(1),kvalitet VARCHAR(15),medium VARCHAR(1),nh VARCHAR(24),objtype VARCHAR(12),vegsperringtype VARCHAR(12),vfradato VARCHAR(8),vnr VARCHAR(9),vpa VARCHAR(13));
SELECT AddGeometryColumn( 'elvegsamferdsel_01','tbl0101Elveg_point','tbl0101Elveg_geom',4326,'POINT',2);

Manually running this gives the following:

SELECT AddGeometryColumn( 'elvegsamferdsel_01','tbl0101Elveg_point','tbl0101Elveg_geom',4326,'POINT',2);
ERROR:  relation "elvegsamferdsel_01.tbl0101Elveg_point" does not exist
CONTEXT:  SQL statement "ALTER TABLE elvegsamferdsel_01."tbl0101Elveg_point" ADD COLUMN "tbl0101Elveg_geom" geometry(Point, 4326)"

Which is quite right because the table was generated with a lowercase name which I believe is because it was not quoted in the create table statement whle the ALTER TABLE statement above uses the quoted table name:

       Schema       |          Name           | Type  |  Owner   
--------------------+-------------------------+-------+----------
 elvegsamferdsel_01 | tbl0101elveg_linestring | table | postgres
 elvegsamferdsel_01 | tbl0101elveg_point      | table | postgres
 elvegsamferdsel_01 | tbl0101elveg_polygon    | table | postgres
(3 rows)

I believe this will also affect the geometry column in the insert statements where it is not quoted:

INSERT INTO elvegsamferdsel_01.tbl0101Elveg_point (datafangstdato,komm,kp,kvalitet,medium,nh,objtype,tbl0101Elveg_geom,vegsperringtype,vfradato,vnr,vpa) VALUES

This is with postgres 11 on Debian 4.9.

Sosicon generated the following psql, which contains a "" instead of NULL for an integer column (snsaksnr, i think)

INSERT INTO sosicon.object_point
            (
                        aar,
                        datafangstdato,
                        komm,
                        navntype,
                        noe,
                        object_geom,
                        objtype,
                        oppdateringsdato,
                        snavn,
                        sndato,
                        snforek,
                        snkilde,
                        snloepenr,
                        snmerk,
                        snmynd,
                        snregdato,
                        snsaksnr,
                        snskrstat,
                        snspraak,
                        sntystat,
                        ssr_id,
                        ssr_objid
            )
            VALUES
            (
                        '',
                        '20120423',
                        '0815',
                        81,
                        '6525012 193783',
                        St_transform(St_geomfromtext('POINT(193783.00000 6525012.00000)',25833),4326),
                        'Navneenhet',
                        '20100227',
                        'Skagerrak',
                        '',
                        '',
                        '',
                        NULL,
                        '',
                        'DEPT',
                        '',
                        '',
                        '',
                        'NO',
                        'H',
                        56,
                        56
            )

Any idea why this would come out as empty string instead of NULL?

Is it possible to get the height values when converting? Se the example coordinates below:

..NØH
664411925 58026891 14602 ...KP 1
..NØH
664411958 58026564 14602
664412279 58026597 14552
664412246 58026923 14552
664411925 58026891 14602 ...KP 1

The coordinates are only parsed as 2D when converting to postgis.

while converting sosi to shp, all the field are converted to text even if they are number. Is the a way to fix that, so the fields stay with the same type?

Some of my SOSI files have an origo different from 0;0. It looks like it's not taken into account so the result data is not at the right place.
Is there a way to fix this?

Is it possible to detect fields that contain only numeric data, and set the appropriate header?
IE. when converting SOSI files, it seems to always use the data type character

Ie.

HØYDE,C,4

Should be

HØYDE,N,4,0

I am trying to use sosicon for selective exports, but there seems to be something wrong with the character set / encoding.

When I run

sosicon -stat NO_N5000_Arealdekke.sos

I get

OBJTYPE                       : COUNT
------------------------------------------
 Arealbrukgrense...............:     1226
 Dataavgrensning...............:      268
 ElvBekk.......................:      138
 FiktivDelelinje...............:      372
 Havflate......................:       65
 Innsj�........................:       49
 Innsj�kant....................:      170
 KantUtsnitt...................:        3
 Kystkontur....................:     1240
 Lufthavn......................:       14
 Skog..........................:      429
 Sn�Isbre......................:       25
 TettBebyggelse................:      125
 �pentOmr�de...................:      756

EDIT
This is from a terminal window with UTF-8 as default charset, as a workaround I set the character encoding to Latin6 (ISO-8859-10) - then I get the desired results. Not sure if this is a bug or not, but it could help to add a note in documentation on how to set the charset for your terminal window.

Incase anyone else needs this, here's how to set the character encoding in terminal required by sosicon/norwegian mapping data:

1. Go to Terminal -> Preferences
2. Go to tab Encodings -> Enable Nordic (ISO Latin 6)
3. Go to Profiles and create a new profile, name it Latin 6 or similar
4. With the new profile selected go to the advanced tab, and set Text encoding to Nordic (ISO Latin 6)
5. Then just start a terminal tab or window with the new profile (Shell -> New window -> select Latin 6 profile)