oemof / oemof-db Goto Github PK

The docs do not seem to have a section for the DB export of results implemented by @henhuy .

psycopg2 which is required by oemof.db fails installing via pip3

Downloading/unpacking psycopg2
  Downloading psycopg2-2.6.1.tar.gz (371kB): 371kB downloaded
  Running setup.py (path:/tmp/pip-build-a798ogk9/psycopg2/setup.py) egg_info for package psycopg2

    Error: pg_config executable not found.

    Please add the directory containing pg_config to the PATH
    or specify the full executable path with the option:

        python setup.py build_ext --pg-config /path/to/pg_config build ...

    or with the pg_config option in 'setup.cfg'.
    Complete output from command python setup.py egg_info:
    running egg_info

creating pip-egg-info/psycopg2.egg-info

writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt

writing pip-egg-info/psycopg2.egg-info/PKG-INFO

writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt

writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'

warning: manifest_maker: standard file '-c' not found



Error: pg_config executable not found.



Please add the directory containing pg_config to the PATH

or specify the full executable path with the option:



    python setup.py build_ext --pg-config /path/to/pg_config build ...



or with the pg_config option in 'setup.cfg'.

Proposed solution is to install two packages via Linux package management:
sudo apt-get install libpq-dev python-dev

This is in general possible. But, same as with #16, I would love to how a solution that is in-line with capabilities of pip. Does anyone know how I can tell pip to install the above package via system package management and this favorably platform independent?

I added oemof.db.__version__ in the 0.0.5 release so that the version number can be queried at runtime. That means we now have two sources of version numbers, the setup.py file and oemof.db.__version__. Due to how oemof/db/__init__.py is written, the latter can not be used in the former. For that reason I should figure out how to use bumpversion to maintain those version numbers consistently or find a different solution to have a single source for this again.

@oemof/oemof-main
Remember to merge changes in the master branch (hotfixes or direct changes on the master branch itself) back into dev.
see http://nvie.com/posts/a-successful-git-branching-model/#hotfix-branches

We should decide on a version numbering policy for oemof.db (and oemof subpackages in different repositories in general.)

This can be done either

1. at installation oemof.db
2. at program start if config file does not exists

Following aspects would improve this

• Ask the user if he/she wants to create the config file
• Ask for the location of config file and provide default location
• Warn about overwriting an existing config file

hi,

I'm getting started with oemof, but I can't figure out how to use oemof.db.
I always get the error "No module named 'oemof.db'.

Thx!

The openego group uses oemof.db to instantiate a DB connection to OEDB. The installation of packages dependent on oemof.db is sometimes hard due to its dependencies.
Therefore, we propose to make some dependencies optional for example shapely (maybe additional other dependencies). @gnn agreed on this.

I would like to discuss with u @uvchik. Maybe we can have lunch next days.

Currently there is no release available at pip. I would like to used oemof.db as dependency in dingo. Shall we release version 0.0.3?

Related issue #14

I have fixed a line in the installation instructions and created a dev branch for this.

Merging back into the master using git merge --no-ff dev somehow didn't work.

What's my mistake here?

Currently, if a database connection is established with the help of oemof.db and the password isn't available in the keyring, program crashes. Make this more convenient by

1. Ask for password entry if password isn't available in keyring
2. Exit program early with information about missing password and provide help to the user how to add the password to the keyring

i need feedin timeseries for a small region in germany.
when i use feedin_pg i get nans for wind_power because there are no wind turbines in this area.
it would be nice to have an option to set the peak power to a "scenario value" or just 1 instead of taking it from energymap

When using oemof.db in a fresh (virtual) environment database access tools fail due to missing keyring backend, see

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/guido/virtualenvs/elesplan_m/lib/python3.4/site-packages/keyring/core.py", line 48, in set_password
    _keyring_backend.set_password(service_name, username, password)
  File "/home/guido/virtualenvs/elesplan_m/lib/python3.4/site-packages/keyring/backends/fail.py", line 18, in get_password
    raise RuntimeError("No recommended backend was available. Install the "
RuntimeError: No recommended backend was available. Install the keyrings.alt package if you want to use the non-recommended backends. See README.rst for details.

I suggest to add keyrings.alt to the dependencies list in order to solve this issue.

@oemof/oemof-db feedback on this is welcome!

First, have a look on oemof's README are make structure similar.

TODOs

• remove/replace announcement of planned oemof release
• Redesign installation section: installation via pip and developer istallation
• Update configuration
• Remove required packages section

Recent keyring updates (8.4ish, 5.6 worked) started errorring out on me, saying something along the lines of "No suitable backends available". Since the case of not having the password in the keyring is already covered in the code, the whole package can be made optional by simply wrapping the import in a try-block.

Installation of oemof.db requires the package shapely to be installed. The installation of shapely via pip3 fails with following message

OSError: Could not find library geos_c or load any of its variants ['libgeos_c.so.1', 'libgeos_c.so']

Available solutions (Workarounds)

• For Linux: sudo apt-get install libgeos-dev
• Windows: please add if you know about (it was something with the installation of libgeos from pre-built binary package)

This enables us to regularly install shapely via pip3 install shapely.

How can we automize this solution/workaround to enable new users to smoothly install oemof.db?

Latest version Tag of oemof.db in git is 0.0.3 whereas latest pip version of oemof.db is 0.0.2.

I assumed both versions should be the same. Is this assumption right?

Due to historical reasons, some table and column names contain prefix stemp_ in results.py.
I'll prepare a fix..

For a pre-release on Dingo we need some additional options in oemof.db that necessitates a new release.
Therefore, I create a list a additions I would to implement. Feel free to add yours!

Add option to specify location of config.ini file in order to dissolve strong dependency on the folder ~/.oemof/

Refers to issue oemof/oemof#100 that was moved from oemof to oemof.db.

After looking at it for a second time, the exception handling introduced in d1ec190 seems funky. Things to note:

• it doesn't have tests, so it's unclear whether it does what it should,
• it doesn't properly work with sections specified as a parameter to connection.

When using oemof.db to establish a database connection, the package psycopg2 is indirectly needed because sqlalchemy requires it. psycopg2 is not automatically installed when installing sqlalchemy though and it is not specified in the install_requires in the setup.py of oemof.db.

I would suggest that we add the extras_require 'postgresql' to sqlalchemy in install_requires so that psycopg2 is installed along with sqlalchemy (see https://github.com/zzzeek/sqlalchemy/blob/master/setup.py).

The install_requires would then look as follows:
install_requires=['sqlalchemy[postgresql] >= 1.0', 'keyring >= 4.0', 'shapely', 'psycopg2'])

@oemof/oemof-db If it's okay with you I would adjust the setup.py accordingly.

Currently the oemof_pg [api documentation]:(http://oemof-pg.readthedocs.org/en/latest/) on RTD is empty but of course it shouldn't be that way. Find out why the apidocs are not build from the available docstrings.

Oemof.db seems to support PostgreSQL only. Yet, the doc-strings in config.py seems do describe the config structure for everything but PostgreSQL (NetCDF, MySQL, ...). This is rather confusing.

It says feedinlib in the root link and the heading of the index page. Clearly it should be oemof_pg.

Updated versions of the dependent packages keyring (10.3.1) and keyrings.alt (2.1) induce import errors.

import oemof.db as db

results in following error

Error initializing plugin Gnome = keyrings.alt.Gnome.
Traceback (most recent call last):
  File "/home/jonathan/python/virtualenvs/dingo/lib/python3.5/site-packages/keyring/backend.py", line 167, in _load_plugins
    init_func = ep.load()
  File "/home/jonathan/python/virtualenvs/dingo/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2302, in load
    return self.resolve()
  File "/home/jonathan/python/virtualenvs/dingo/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2308, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/home/jonathan/python/virtualenvs/dingo/lib/python3.5/site-packages/keyrings/alt/Gnome.py", line 11, in <module>
    from keyring.py27compat import unicode_str
ImportError: cannot import name 'unicode_str'

A quick work-around is to limit versions of these packages to

• keyring 10.3.0
• keyrings.alt 2.0

The actual problem is something related to the SecretStorage backend or similar. Haven't figured out this in detail. This is for some reason only available in Gnome.
Thus, maybe the above mentioned work-around may work as solution to fix this issue.

Relying on the configuration file being in the home directory makes some things (e.g. behaviour on missing sections) difficult to test as it's impossible to know what's in the configuration file at test time. Being able to specify a differen't location (or something similar) would make testing things easier (and could have other benefits).
Fortunately there are a few ways of implementing this without breaking backwards compatibility:

• implement a way to override configuration settings from within python code,
• use a default (documented) set of locations to search for configuration files.

Namespaces of dev and master branch are different. In order to avoid confusion it would be helpful to align these both. For example name folder oemof_pg to oemof. This would enable user to run two_regions_example_opt.py with checked out master branch of oemof.db.