Posgres connection with custom db name and MD5 auth fails about kartotherian HOT 17 CLOSED

@benstadin, are you using osm-bright source? If so, our own production uses this approach to dynamically change the source's XML and inject proper user name and password. Simply replace the {...} with your values, and it should work. So in your case it would be something like this. (although I haven't used MD5 auth myself, so there might be something else that postgress requires)

user: username
password: md5hashString

from kartotherian.

This was the first thing I tried to change, but the message when starting the server indicates that it was still trying to connect to a db gis with a user gis. I've basically followed all install steps, though I'm on Ubuntu 10.04.

I've basically done the following:

• Imported an extract into a Postgres db gis_bw
• cp config.sample.yaml config.yaml (and point the source to sources.yaml)
• cp sources.prod.gen.yaml sources.yaml, and changed user / pw settings there
• started server via: node server.js -c config.sample.yaml

from kartotherian.

@benstadin, don't use source.prod.gen.yaml - it might confuse the debugging. Use the sources.sample.yaml, and add just these lines on line 5 (indent xmlSetDataSource by 2 spaces)

  xmlSetDataSource:
    set:
      host: localhost
      user: ...
      password: ...

When you run the service, note if it shows "Updating layer ..." log lines.

from kartotherian.

Thanks, one step closer. I also had to install postgis-vt-util for missing SQL functions (Z()).

There is a new issue now about some missing relation:
"Postgis Plugin: ERROR: relation "admin" does not exist"

from kartotherian.

@benstadin, i fixed the doc - cc86e79

from kartotherian.

I can start the server now, though I get a few errors after startup. It is still hanging at updating the road_label table since a few minutes. Is it normal?

This is the console log:
ben@denktablette:~/dev/kartotherian$ npm start

[email protected] start /home/ben/dev/kartotherian
service-runner

{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":40,"msg":"Could not load the spec: Error: ENOENT, no such file or directory '/home/ben/dev/kartotherian/spec.yaml'","time":"2015-09-21T20:23:35.939Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Loaded tilelive-bridge","time":"2015-09-21T20:23:36.003Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Failed to load tilelive-file: Cannot find module 'tilelive-file' ","time":"2015-09-21T20:23:36.005Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Loaded tilelive-vector","time":"2015-09-21T20:23:36.113Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Loaded kartotherian-autogen","time":"2015-09-21T20:23:36.135Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Loaded kartotherian-demultiplexer","time":"2015-09-21T20:23:36.159Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Loaded kartotherian-overzoom","time":"2015-09-21T20:23:36.187Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Loaded kartotherian-cassandra","time":"2015-09-21T20:23:36.256Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Loaded kartotherian-layermixer","time":"2015-09-21T20:23:36.286Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Loading variables from the config file","time":"2015-09-21T20:23:36.288Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Loading sources from /home/ben/dev/kartotherian/sources.yaml","time":"2015-09-21T20:23:36.291Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Worker 22774 listening on 0.0.0.0:6533","time":"2015-09-21T20:23:36.291Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Updating layer landuse","time":"2015-09-21T20:23:36.330Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Updating layer waterway","time":"2015-09-21T20:23:36.330Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Updating layer water","time":"2015-09-21T20:23:36.330Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Updating layer aeroway","time":"2015-09-21T20:23:36.330Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Updating layer building","time":"2015-09-21T20:23:36.331Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Updating layer road","time":"2015-09-21T20:23:36.331Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Updating layer admin","time":"2015-09-21T20:23:36.331Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Updating layer country_label","time":"2015-09-21T20:23:36.331Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Updating layer place_label","time":"2015-09-21T20:23:36.331Z","v":0}
{"name":"kartotherian","hostname":"denktablette","pid":22774,"level":30,"msg":"Updating layer road_label","time":"2015-09-21T20:23:36.331Z","v":0}

from kartotherian.

Server is running now. But this is what I get when viewing the map:

"err":{"message":"Unknown source","name":"kartotherian","stack":"Err: Unknown source\n at /home/ben/dev/kartotherian/routes/kartotherian.js:121:19\n at tryCatcher (/home/ben/dev/kartotherian/node_modules/bluebird/js/main/util.js:24:31)\ [snip]

This is my config.yaml:

# Number of worker processes to spawn.
# Set to 0 to run everything in a single process without clustering.
# Use 'ncpu' to run as many workers as there are CPU units
num_workers: 0

# Log error messages and gracefully restart a worker if v8 reports that it
# uses more heap (note: not RSS) than this many mb.
worker_heap_limit_mb: 250

# Logger info
logging:
  level: trace

# Statsd metrics reporter
metrics:
  type: info

services:
  - name: kartotherian
    module: ./app.js


    conf:
      port: 6533
      # Comment this out to allow connections from everywhere
      interface: localhost

      variables: {}
      sources: sources.yaml

And this my sources.yaml

gen:
  uri: bridge://
  xml:
    npm: ["osm-bright-source", "data.xml"]
  xmlSetDataSource:
    set:
      host: localhost
      dbname: gis_bw
      type: postgis
      user: 'mapuser'
      password: 'xxxxxxx'
oz:
  uri: overzoom://
  params:
    source: {ref: gen}

osm:
  public: true
  uri: vector://
  formats: [png,json,headers,svg,jpeg]
  xml:
    npm: ["osm-bright-style", "project.xml"]
  xmlSetParams:
    source: {ref: oz}
  xmlExceptLayers: ['water']

from kartotherian.

Try to browse it with http://localhost:.../?s=osm -- by default, the code in static uses "osm-intl" style, which you don't have. You could either define it in the source, or change the static files. I should probably change the sample source to use 'osm-intl' instead of 'osm'

from kartotherian.

BTW, your log looks fine - everything is ok - the spec is not finished yet, and the tilelive-file is not loaded but requested (just in case you add it via npm install tilelive-file

from kartotherian.

Fixed in 6ea5d2e

from kartotherian.

Thanks, works fine now.
Another unrelated question: How can I play with the styles? I'd like to fiddle with the osm bright style, but my changes to the related style files inside node_modules/osm-bright-style aren't reflected. Maybe it is because I'm misunderstanding npm, but it would be helpful if it was possible to tinker with styling in an easy way.

from kartotherian.

There is a very easy way -- https://github.com/kartotherian/osm-bright.tm2/blob/master/README.md

from kartotherian.

Where do I need to set this source property? I tried it in the source.yaml file without luck:

osm:
  public: true
  uri: vector://
#  scales: [1.3, 1.5, 2, 2.6, 3]
  formats: [png,json,headers,svg,jpeg]
  xml:
   # npm: ["osm-bright-style", "project.xml"]
    source: "tmsource:///home/ben/dev/osm-bright.tm2"
  xmlSetParams:
    source: {ref: oz}
 # xmlExceptLayers: ['water']

from kartotherian.

You can set the pbfinfo data inside the cassandra's storage -- that's how https://maps.wikimedia.org/osm/pbfinfo.json works, but I don't think i have the code to do it from the tile generator. Might need to work on that at some point - should be fairly easy.

from kartotherian.

Also, you don't actually need to use any of the kartotherian to edit the styles and/or sources -- simply edit the kartotherian/node_modules/osm-bright-style/project.yml - set source to point to "tmsource:///home/user/kartotherian/node_modules/osm-bright-source" (adjust the path), and open it in the mapbox studio (0.2.8 or later)

from kartotherian.

I'll not be able to use Mapbox Studio for what I intend to do unfortunately (I'll trying to create styles on the fly at some point).

from kartotherian.

yeah, for that you will need to code up something funky. Kind of an unusual request. Please start a new thread if you wish so we could discuss it.

from kartotherian.