Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• Hide the above functions (prepending an underscore)
• use/define add_input and add_output?

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/40

Originally reported by: AiiDA Team (Bitbucket: aiida_team, GitHub: Unknown)

Create a test script for operating with entities using ORM.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/1

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/24

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

Think to a way to automatically set new attributes of stored nodes.
This should be user dependent (e..g, files stored in ~/.aiida), e.g. with a function that is called automatically every time a node of a given type is stored.
Manage also correctly the exception handling: even if there are errors, a warning should be issued but the node should be saved anyway!

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/31

Originally reported by: AiiDA Team (Bitbucket: aiida_team, GitHub: Unknown)

Define API for writing code plugins.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/16

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

By overriding getattr method of the django table.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/14

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

Test how paramiko works, and possibly make it the default way of connecting through ssh in python

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/6

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/23

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

(instead of merging both in aiida_deamon.log)

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/49

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

Issue: some schedulers allow to have a decently complete output, and to filter it by providing a list of jobs (PBSPro, SLURM, PBS, ...)

Others (SGE) instead allow to filter only by user, and not by jobid (unless one wants some completely different output with unuseful information).

Solution: implement an attribute of the plugin to tell which features are available, and use the available features.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/47

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• Recheck all the places where a ValueError is raised, and convert it to TypeError where appropriate
• Fix tests (and documentation)

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/37

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

E.g. when one calls the Calculation.set_state() function; e.g. an attribute named state_changed_to_XXX
Rethink if it is possible to add the date/times returned by the scheduler in the DB, or simply leave them in the json.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/39

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

Prepare the template, populate with first instructions to install, prepare makefile to create doc (probably automatically done by the sphinx scripts).
Start to write first docstrings.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/3

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

implement verdi gotocluster to directly ssh to the folder of the specific calculation.
Note: what to do depends on the transport. Improve the transport to have an API to return the command to execute.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/25

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• implement the -V option of most schedulers (i.e., inheritance of the environment variables from the launching environment) both in the plugins + options to set this flag as methods of a calculation
• Check if writing 'export AAA=BBB' in the scheduler script itself is sufficient, and if the environment variables are obtained by all processors or only by the first one.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/36

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• Implement the output parser for Quantum Espresso - pw
• Configure it and try it out
• Write documentation on the output parser API interface

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/18

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• Proposal: verdi computer add to add a computer, verdi computer configure to setup the authentication (AuthInfo table)
• command line interactive interface
• Automatic filling up of the authentication info, similar to what is currently done in the submission example

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/19

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• Restructure the docs folder, moving out of the way old documentation
• Have the automatic documentation from the docstrings
• Add a few sections (developers, users)
• Under developers, prepare the documentation on how to write a transport, scheduler, code (input and output) plugin + developer conventions (function names, code style, ...)
• Under users, write how to setup the code + prepare a stub for a first tutorial

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/26

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

Check if the lock used in the poller hangs until the lock is released, or simply the current execution is skipped.

The second behavior is expected, to avoid that a long list of pending runs waits if one retrieve takes considerable time.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/45

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

In SGE, you don't specify number of nodes and number of cpus per node, but the total # of cpus (slots) and the parallel environment (a string, as defined by the cluster admin).

We should find a way to uniform the way in which you specify the resources.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/50

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

Use django-south ASAP for the schema management (unless specific issues arise).

Use south to populate the DB with start values for known tables, as statuses, code names, ...

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/2

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

As soon as we will start working with AIDA, we'll end up with tens/hundreds/... of groups, which may become unmanageable. We must implement a folder like structure.
Approaches:

• Use a / as path separator, and then at the code-level manage the folders. Disadvantages: difficult to query/browse, must address 'empty' folders (as if I define the group named 'a/b' but no group 'a'.
• Use dependencies between groups, with 'None' dependency meaning the root level, and putting a unique_together constraint for name+parent. Advantages: easy to get parent/children folders by querying. Disadvantage: must explicitly check that there are no 'loop' dependencies.
• Other (better) approaches??

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/8

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• Better method names:
  • get_content_list: use listdir as in os
  • insert_path: e.g. put
  • remove_path: e.g. remove
• Better defaults:
  • get_abs_path should by default take relpath='.'
• get_file_content:
  • add in aiida.common.Folder because it will need to do different things for different storage backends
• deprecate get_file_content in aiida.orm.folder.FolderData

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/42

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

Think to how to autogroup or tag calculations launched within the same script (e.g. I write a script to submit 100 calculations, and they are automatically tagged in a group with name, e.g., autogroups/YYYYMMDD-HHMMSS or something else)

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/46

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

Now, in every subclass init method, one has to rewrite some logic to manage the case in which a uuid is passed as argument.

Think if there is a better way of managing the reload from UUID.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/34

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

Test tastypie+slumber for complex queries.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/15

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

Now, if a parameter data is a recursive dictionary, as

#!python

{ 'CONTROL' : 
  {'calculation': 'scf',
   'verbosity': 'high'}
}

only the first-level dictionary is exploded in attributes/properties, while each inner dictionary is simply stored as json, making queries difficult/impossible to do.

Implement querying also of inner fields, instead

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/28

Originally reported by: AiiDA Team (Bitbucket: aiida_team, GitHub: Unknown)

Which attributes should entity objects be allowed to have? And how do we manage them?
There are three types:

1. Mandatory attributes like Calc.user which are enforced on DB and ORM level.
2. Optional for convenience like calc1.cutoff. These may be temporary input parameters that the user may only need for template / input file generation or script control flow logic.
3. Attributes like calc1.energy thata user may want to store in the DB (CalcAttrNum). Somehow one would specify a list of attributes to be saved and their datatypes.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/12

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/27

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

I.e., the backed-up repository of files referenced by the LOCAL_REPOSITORY variable in settings.py

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/17

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

The current directory structure mixes a lot the general library modules of aida (e.g. the submission scripts, the parsers, ...) with the code which is related to a given aida instance (i.e. the settings.py which points to a specific database etc.)

In principle, the latter //shouldn't// be put in the PYTHONPATH, but only the former libraries.

In the remote case that a user wants to have 2 aida instances on the same machine, the libraries (aidalib, aidasrv.submitter, ...) should be on the python path and referenced by both aida instances, while the settings.py should not be on the python path.

To discuss soon, because then one has to rename every module import!

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/10

Originally reported by: Andrea Cepellotti (Bitbucket: acepellotti, GitHub: cepellotti)

Enhance the QE input plugin, so that a user can append to the list of default calcinfo.retrieve_list, also personalized values, like bands and charge density files.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/32

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

If a --devel option os passed to verdi install, a few more parameters are asked. For the moment, the new one should be:
```confs.get('use_inmemory_sqlite_for_tests')``
that, if True (default), uses a sqlite db for tests, resident in memory; if False, uses the same DB backend that is normally configured.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/30

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

Nicola points out that there are similar projects with similar names.
We should probably also find out a domain name for the 'official' aida page.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/5

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

Beside the logger, store as 'comments' of a DbNode, with user=None, AiiDA messages (e.g. problem retrieving files, etc.).

• Allow for user=None
• Implement this in the execmanager
• create useful interface to read the comments of a calculation

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/33

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

Check in the documentation if writing
#!/bin/bash
as the first line of the submission script is sufficient to choose the shell, or if it would be better to use
#PBS -S /bin/bash

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/35

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

Rewrite RemoteData objects to use the computer attribute field in the database to point to a remote computer, instead of storing it internally in an incorrect way

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/21

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

e.g. to get an input with a given label, or of a given type; improve the API itself

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/41

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• Each transport should define a list of exceptions that, when raised, mean that there was a connection error (e.g. socket.timeout and probably paramiko.SSHException for ssh).
• check for these exceptions in the execmanager and manage them correctly (num_retries, message to user, ...)

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/43

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• Write tests for the input plugins of PW
• move the common code to a generic class, use multiple inheritance to use the same codebase both for PW and CP
• write documentation on how the input should be written for PW and CP + the settings node

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/22

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

On some clusters, the settings for 'modules' is in /etc/profile, that is not sourced if a command is run via ssh, leading to errors.

• Understand what is the best way to fix it
• in verdi computer add also a test command that tries to get the output of the queue, as a test of the good functionality
• add a FAQ saying what to do if there are such problems
• add a FAQ to say that the .bashrc should not produce any output

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/20

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• In verdi install, at the end set the correct permissions on ~/.aiida (write a function to set the permissions)
• Possibly: when loading the settings.py, stop if the permissions are incorrect

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/29

Originally reported by: Andrea Cepellotti (Bitbucket: acepellotti, GitHub: cepellotti)

The most recent PostgreSQL v9.2 has added a JSON field. Investigate if it is worth using this feature.

It seems that now it has limited capabilities, that will be extended in the next versions. Check these links as reference:

http://www.postgresql.org/about/news/1415/
http://stackoverflow.com/questions/10560394/how-do-i-query-using-fields-inside-the-new-postgresql-json-datatype
http://stackoverflow.com/questions/12402630/how-do-i-use-postgresql-9-2-json-data-type-in-django

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/7

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

See also
here
and the comment here for its use with South

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/9

Originally reported by: marco dorigo (Bitbucket: mdorigo, GitHub: Unknown)

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/48

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/38

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

Allow to have links to other calculations (or structures etc., to see what we really need; I think calculations at minimum) in comments to calculations.

For instance, if the string #123 appears in a comment, this will point to calculation with ID 123.

Some issues:

• find a suitable way of distinguish links to different items (calc, struct, ...)
• the web interface (at least) must provide real links
• possibly, a way to query for calculations containing links to a given structure? [new many2many field?]
• IMPORTANT: probably we shouldn't link to the ID but to the UUID so that we can safely export data from the database. The best: the user inputs something as #123, this is automatic converted upon save in the DB as M2M link + converted to a link to the correct UUID.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/11

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

It is useful to have a script that can be run by the user to verify if the database, local repository etc. are in a consistent state.

Since what has to be checked will be clear while we develop the code, I open a wiki page with a list of things to check.
Please add items to that page while you develop and realize that something could be inconsistent.

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/13

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• regenerate the Django secret key, store it in ~/.aiida?
• configure correctly the time zone (dynamically from verdi install) otherwise all dates stored in the DB are wrong! (likely saved in UTF time or similar)

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/44

Originally reported by: Giovanni Pizzi (Bitbucket: pizzi, GitHub: giovannipizzi)

• script to submit jobs
• management of submission broken down in all elementary steps, as:
  • scheduler management
  • code plugin management
  • ...
• test suites? Docs?

• Bitbucket: https://bitbucket.org/aiida_team/aiida_core/issue/4