From [email protected] on April 21, 2012 17:11:06

This is not so much a problem, but more a question. Since there isn't really a forum or anything else to ask questions I just thought I'd post it on here.

Since I am trying to understand how to work with deap I like to read into the source code to see what deap actually does for me. On the page about Evolution Strategies Basics ( http://deap.gel.ulaval.ca/doc/0.8/examples/es_fctmin.html ) I read that "The following generation function takes as argument the class of individual to instantiate, icls. It also takes the class of strategy to use as strategy, scls."

I searched around for these classes, but I can't find them anywhere. Since the example file works perfectly well I just wondered where these classes come from so that I can look into what they exactly do.

Any tips on where to find icls and scls would be very welcome.

Kind regards

Original issue: http://code.google.com/p/deap/issues/detail?id=9

From marc.andre.gardner on August 13, 2010 15:04:52

The Python parser has a depth limit somewhere between 92 and 99 when parsing an expression. For instance, eval("("_100+"3"+")"_100) will fail with a MemoryError exception thrown.

And so there's also a global tree height limit at almost 100. As this limit is pretty high, this should not be a problem.

The real issue came from the evaluation of the GP trees, when the built-in eval function is used (symbreg, parity, etc.).

When an individual is stringified, Python use the str() and repr() functions of base.Tree.
These functions create an object which looks like :
"eap.creator.Individual([mul, eap.creator.Individual([mul, eap.creator.Individual([sin, ...

In this case, both ( and [ are put onto the parser stack -- and so the real practical tree height limit is rather between 45 and 50, which is high but, I think, not unrealistic : a simple symbreg run with 50 generations can easily reach that without bloat control.

One quick fix could be to simply use a different function to create the evaluation function, like :

def createEvalFunc(self):
return list(i.createEvalFunc() for i in self)

in base.Tree or gp.Tree (with the correspondant function in NodeProxy returning self.obj).

which actually return something like [mul, x, [safeDiv, x, 1]], and does not seem to slow down the evaluation.

Also, another question would be to determine if it's would be worthwhile to add a height check in the stringify function : although it would allow EAP to warn user and/or use different evaluation mode, it may add a significant overhead on the evaluation of large population (by calling tree.height on each individual)...

Original issue: http://code.google.com/p/deap/issues/detail?id=4

this is more a suggestion than a bug-report.

A lot of projects use Semantic Versioning as versioning scheme. Following it has some immediate benefits for people using a lot of frameworks/libraries (e.g., not having to learn a special meaning for the version of each of those libs).

I just thought i'd bring this up as the porting guide seems to imply you're using some other versioning scheme.

I am developing a multi-dimensional symbolic regression program, and it seems to work pretty much fine when I have three input variables, but when I have 10 input variables (plus some constants to be solved for), it dies (see below). Is there a limit to how many inputs can be set using pset.renameArguments(ARG0='x_0')?

Traceback (most recent call last):
  File "pysr.py", line 235, in <module>
    main(None, xvecs, yvec)
  File "pysr.py", line 183, in main
    halloffame=hof, verbose=True)
  File "C:\Users\ihb\AppData\Local\Continuum\Miniconda\lib\site-packages\deap\algorithms.py", line 147, in eaSimple
    fitnesses = toolbox.map(toolbox.evaluate, invalid_ind)
  File "pysr.py", line 152, in evalInvSqrt
    disp=False)
  File "C:\Users\ihb\AppData\Local\Continuum\Miniconda\lib\site-packages\scipy\optimize\optimize.py", line 374, in fmin
    res = _minimize_neldermead(func, x0, args, callback=callback, **opts)
  File "C:\Users\ihb\AppData\Local\Continuum\Miniconda\lib\site-packages\scipy\optimize\optimize.py", line 433, in _minimize_neldermead
    fsim[0] = func(x0)
  File "C:\Users\ihb\AppData\Local\Continuum\Miniconda\lib\site-packages\scipy\optimize\optimize.py", line 282, in function_wrapper
    return function(*(wrapper_args + args))
  File "pysr.py", line 113, in evalParams
    errsum = math.fsum(sqerrors)
  File "pysr.py", line 110, in <genexpr>
    sqerrors = ((func(*(list(IO[1::]) + list(params))) - IO[0])**2 for IO in zip(yvec, *xvecs))
TypeError: <lambda>() takes exactly 5 arguments (10 given)

By cmd-ntrf

Currently, DEAP handles constraints by checking that bounds are respected through either specialized operators like mutPolynomialBounded and cxSimulatedBinaryBounded or decorators such as staticLimit (gp) and checkBounds (see example ga/kurswafct.py). Through these mechanisms, we ensure that every individuals produced and evaluated will be valid, respecting constraints.

It is however not always possible to restrict the generated individuals to the feasible domain. In which case, it is required to assess the feasibility of solutions. While the concept of fitness evaluation is well defined, it is less clear how the concept of feasibility assessment can be integrated in DEAP's examples and algorithms.

Kurpati et al. (2002) list three assumptions on constraints handling in a multi-objective context, :
(i) In a given population, feasible solutions are preferred over infeasible solutions, i.e. feasible solutions should have a better rank than the infeasible ones.
(ii) The amount of infeasibility (or the extent of constraint violation) is an important piece of information and should not be ignored while handling constraints.
(iii) The number of violated constraints is also an important piece of information and should be taken into consideration while handling constraints.

These guidelines could serve as design principles for integration of feasibility assessment in DEAP.

Kurpati, A. and Azarm, S. and Wu, J. "Constraint handling improvements for multiobjective genetic algorithms", Struct Multidisc Optim 23, pp. 204–213, 2002

From [email protected] on July 17, 2010 10:03:43

Suppose you have created a function that takes mandatory arguments, for example
{{{
def list_generator(mu, sigma, items):
"""A function to create lists of gaussian length"""
length = random.gauss(mu, sigma)
return [random.sample(items) for i in xrange(length)]
}}}

Then you use it in the toolbox to initialize a class created by the creator. Note the missing arguments for the function list_generator.
{{{
creator.create("Individual", list)
tools.register("individual", creator.Individual, content_init=list_generator)
}}}

The try/except at line 51 of the creator module will eat the exception raised when calling the list_generator function without arguments. The result of calling {tools.individual()} is an empty individual instead of a error.

Original issue: http://code.google.com/p/deap/issues/detail?id=3

See thread : https://groups.google.com/forum/#!forum/deap-users

From [email protected] on June 30, 2012 16:10:45

.

Original issue: http://code.google.com/p/deap/issues/detail?id=10

I am getting this issue when trying to install the package in VS2013 (PTVS). I am also getting this from the command line as well. I am using Python 3.4.

Installing 'deap'
Collecting deap
  Using cached deap-1.0.2.post1.tar.gz
    warning: pypandoc module not found, could not convert Markdown to RST
    Traceback (most recent call last):
      File "<string>", line 20, in <module>
      File "C:\DOCUME~1\USER\LOCALS~1\Temp\pip-build-a1bo2jcu\deap\setup.py", line 22, in <module>
        long_description=read_md('README.md'),
      File "C:\DOCUME~1\USER\LOCALS~1\Temp\pip-build-a1bo2jcu\deap\setup.py", line 13, in <lambda>
        read_md = lambda f: open(f, 'r').read()
    FileNotFoundError: [Errno 2] No such file or directory: 'README.md'
    Complete output from command python setup.py egg_info:
    warning: pypandoc module not found, could not convert Markdown to RST

    Traceback (most recent call last):

      File "<string>", line 20, in <module>

      File "C:\DOCUME~1\USER\LOCALS~1\Temp\pip-build-a1bo2jcu\deap\setup.py", line 22, in <module>

        long_description=read_md('README.md'),

      File "C:\DOCUME~1\USER\LOCALS~1\Temp\pip-build-a1bo2jcu\deap\setup.py", line 13, in <lambda>

        read_md = lambda f: open(f, 'r').read()

    FileNotFoundError: [Errno 2] No such file or directory: 'README.md'

    ----------------------------------------
    Command "python setup.py egg_info" failed with error code 1 in C:\DOCUME~1\USER\LOCALS~1\Temp\pip-build-a1bo2jcu\deap
'deap' failed to install. Exit code: 1

From [email protected] on March 03, 2014 08:28:03

The arity attribute is inherited from Terminal, but it is not set as a property. Thus when accessed it returns an object of type property which is a bit confusing. Revision 8c15ba242d9a , by MA fixed this issue in dev. It should be backported for 1.0.1

Original issue: http://code.google.com/p/deap/issues/detail?id=27

Would it be possible to allow the user to specify the variator strategy in the evolutionary algorithms (e.g. eaMuPlusLambda) ?
At the moment in eaMuPlusLambda the variator strategy is hard coded as varOr.
This could e.g. be achieved by letting the user register a 'variate_strategy' (name to be chosen) function (and fall back to the default behavior in case no such function was registered).
Thanks.

From [email protected] on March 06, 2014 18:42:19

What steps will reproduce the problem? 1. Run knapsack example 2. 3. What is the expected output? What do you see instead? Expect example output. Python reports "ValueError: too many values to unpack" instead. What version of the product are you using? On what operating system? Deap version 1.0.0 on Python 2.7 on Windows 8 Please provide any additional information below. Line 221 of algorithms.varOR is:
ind, = toolbox.mutate(ind)

Removing the comma after ind fixes the issue

Original issue: http://code.google.com/p/deap/issues/detail?id=29

From felix.antoine.fortin on August 26, 2013 14:08:39

I am currently reviewing our code to create Python code from DEAP's tree while working with SCOOP developper on pickling of lambda function and I reminded a small problem we currently have in DEAP.

When creating a lambda function from a tree, we use Python eval function, with two arguments: the program in the form of a string previously converted with gp.stringify and a copy of the primitive set context dictionary. We create a copy of pset context dictionary because as specified by Python documentation: " If the globals dictionary is present and lacks ‘builtins’, the current globals are copied into globals before expression is parsed". Since, we do not want the pset context dictionary to be polluted by unused builtins, we do a copy before passing the dictionary to eval.

Now, it appears that if the Primitive Set context would contain the keyword "builtins", the context would be not be updated with unused builtins. So, in PrimitiveSet.init we could initialize the context dictionary with a key "builtins" associated to None (or no value). This would avoid the copy the pset dictionary at every call of lambdify, even though after a quick test, it does not seem to provide any gain in execution speed.

We could also provide an empty dictionary for the "locals" argument of eval(), since the lambda evaluation has no use for locally defined variables provided by locals().

I have included a patch that could be applied on 1.0.0rc2 and included into next 1.0.0 release candidate.

Attachment: patch_gp_eval.diff

Original issue: http://code.google.com/p/deap/issues/detail?id=22

From felix.antoine.fortin on March 15, 2013 12:15:36

Purpose of code changes:
Enhance readability of code regarding the isDominated function and allow users to implement a dominates method different than Pareto dominance.

I have also integrated the hash and bool method. When reviewing my code changes, please focus on: Multi-objective operators using the dominates method instead of isDominated.

After the review, I'll merge this into:
dev

Attachment: base.py tools.py

Original issue: http://code.google.com/p/deap/issues/detail?id=17

I've been using DEAP for a simple genetic algorithm using NSGA-II. For large populations and many applications of the cxSimulatedBinaryBounded operator, I eventually end up with a TypeError:

TypeError: unorderable types: complex() < float()

The culprit is the following line:

c1 = min(max(c1, xl), xu)

I assume that for certain values, c1 ends up complex (using Python 3.4.1). User error? Bug? Should there be some safeguard that prevents this? It's unfortunate because it (stochastically) kills large jobs.

From [email protected] on June 20, 2011 04:59:15

What steps will reproduce the problem? 1. Use toolbox.migRing with toolbox.selRandom Please provide any additional information below. Remember not to use selectors that might choose twice the same element, otherwise you'll get:

indx = populations[to_deme].index(immigrant)
ValueError: list.index(x): x not in list

Original issue: http://code.google.com/p/deap/issues/detail?id=6

From [email protected] on August 06, 2012 12:09:20

What version of the product are you using? On what operating system? I am using a Debian unstable with

ii python-deap 0.8.2-1 all Distributed Evolutionary Algorithms in Python
ii python-deap-doc 0.8.2-1 all Distributed Evolutionary Algorithms in Python - documentation Please provide any additional information below. picca@xxxxxx:~/Projets/dadimodo$ mpirun -n 1 ./bootstrap.py deap_test /home/picca/Projets/dadimodo/doc/example1/test.conf
Executing deap_test from source checkout

1. Patched sys.path with /home/picca/Projets/dadimodo
2. Patched sys.path with /home/picca/Projets/dadimodo/scripts
   ('03. patch the sys.argv : ', ['deap_test', '/home/picca/Projets/dadimodo/doc/example1/test.conf'])
3. Executing deap_test.main()
   checking cycle parameters

Creating required directory structure
Transcripting initial PDB files to INITIAL directory...
...done.
Calling pb_config._crop_datafile() from the SAXS_Fitness constructor...
possible cropping of the initial data
initial file: /home/picca/Projets/dadimodo/doc/example1/SAXS_data.dat
requested Q-range: 0.004400, 0.440000
...
Opening your initial file...
reading the initial data file...
number of initial data points found: 100
extremal x-values found: xmin=0.004400 xmax=0.440000
no need to crop the data file
Q_min, Q_max and data file contents are consistent
Child returned 0
reading crysol output ...
Traceback (most recent call last):
File "./bootstrap.py", line 34, in
module.main()
File "/home/picca/Projets/dadimodo/scripts/deap_test.py", line 89, in main
fitnesses = toolbox.map(toolbox.evaluate, population)
File "/usr/lib/pymodules/python2.7/deap/dtm/manager.py", line 1029, in map
currentId = cThread.tid
AttributeError: '_MainThread' object has no attribute 'tid'

So it seems that dtm does not work with only one core.

If I use mpirun -n 2 it is ok.

Now my script should work with mpi and without.
Is there a simple way from the script to determine if the program was starter from mpirun or not ?

thanks

Frédéric

Original issue: http://code.google.com/p/deap/issues/detail?id=11

Using the example spambase.py the string
"add(mul(protectedDiv(ARG15,-82.5), protectedDiv(ARG24, ARG24)), ARG26)"
fails to be parsed by from_string()

gp.py line 141 rasises the error:
"Terminal { -82.5} type {float } does not match the expected one: {object}

This is using Python 2.7.6 on Windows 7 x 64 and DEAP 1.0.1

From [email protected] on January 24, 2014 01:25:17

What steps will reproduce the problem? ----------------------------------

1. Use a source encoding utf-8 (as typing code in french), or the command-line below:

$ python -c 'import deap;print deap.version'
1.0
$ python -c 'from deap import creator, base;creator.create(u"Fit", base.Fitness, weights=(-1.0,))'
Traceback (most recent call last):
File "", line 1, in
File "/Users/nino/Library/Python/2.7/lib/python/site-packages/deap/creator.py", line 152, in create
objtype = type(name, (base,), dict_cls)
TypeError: type() argument 1 must be string, not unicode
$ python -c 'from deap import creator, base;creator.create(str(u"Fit"), base.Fitness, weights=(-1.0,))'
$ What is the expected output? What do you see instead? ----------------------------------------------

Expect no error; see TypeError. What version of the product are you using? On what operating system? version 0.9.1 and 1.0.0rc2 both behave the same way. On macosx with macports python 2.7.6 and
pip-installed deap. Please provide any additional information below. -----------------------------------------

In the incriminated line (152 of creator.py) it would be simple to use

objtype = type(str(name), (base,), dict_cls)

instead of

objtype = type(name, (base,), dict_cls)

which is the workaround I use now. Thanks for the very good work.

Original issue: http://code.google.com/p/deap/issues/detail?id=23

Very minor suggestion, but in the current version of the gp module, there appear to be minor inconsistencies in the API for addEphemeralConstant vs. addTerminal or addPrimitive:

def addEphemeralConstant(self, name, ephemeral, ret_type):

# versus...
def addPrimitive(self, primitive, in_types, ret_type, name=None):
def addTerminal(self, terminal, ret_type, name=None):

I don't know if there's a principled reason for this, but if not, it might be helpful to keep the arguments in the same order across the API. Would be happy to submit a PR.

Reported by cgranade, Jun 4 (6 days ago)
In the documentation posted at [0], it is not made clear which selection operators are single-objective, which ones treat the fitness vector in a lexographic fashion, and which are useful in MOEA contexts. For instance, selTournament seems to act lexographically, as it uses the sorted built-in with a tuple-valued fitness as the key, but this is not made clear in the documentation at [1].

Links:
[0] http://deap.gel.ulaval.ca/doc/default/api/tools.html
[1] http://deap.gel.ulaval.ca/doc/default/api/tools.html#deap.tools.selTournament

I cannot make spambase example run correctly. I get the following error (using version 1.0 installed from pip, with python3).

Traceback (most recent call last):
  File "/usr/lib64/python3.4/random.py", line 253, in choice
    i = self._randbelow(len(seq))
  File "/usr/lib64/python3.4/random.py", line 230, in _randbelow
    r = getrandbits(k)          # 0 <= r < 2**k
ValueError: number of bits must be greater than zero

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/username/.local/lib/python3.4/site-packages/deap/gp.py", line 584, in generate
    prim = random.choice(pset.primitives[type_])
  File "/usr/lib64/python3.4/random.py", line 255, in choice
    raise IndexError('Cannot choose from an empty sequence')
IndexError: Cannot choose from an empty sequence

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "spamthing.py", line 110, in <module>
    main()
  File "spamthing.py", line 97, in main
    pop = toolbox.population(n=100)
  File "/home/username/.local/lib/python3.4/site-packages/deap/tools/init.py", line 23, in initRepeat
    return container(func() for _ in range(n))
  File "/home/username/.local/lib/python3.4/site-packages/deap/tools/init.py", line 23, in <genexpr>
    return container(func() for _ in range(n))
  File "/home/username/.local/lib/python3.4/site-packages/deap/tools/init.py", line 49, in initIterate
    return container(generator())
  File "/home/username/.local/lib/python3.4/site-packages/deap/gp.py", line 539, in genHalfAndHalf
    return method(pset, min_, max_, type_)
  File "/home/username/.local/lib/python3.4/site-packages/deap/gp.py", line 524, in genGrow
    return generate(pset, min_, max_, condition, type_)
  File "/home/username/.local/lib/python3.4/site-packages/deap/gp.py", line 589, in generate
    "none available." % (type_,)).with_traceback(traceback)
  File "/home/username/.local/lib/python3.4/site-packages/deap/gp.py", line 584, in generate
    prim = random.choice(pset.primitives[type_])
  File "/usr/lib64/python3.4/random.py", line 255, in choice
    raise IndexError('Cannot choose from an empty sequence')
IndexError: The gp.generate function tried to add a primitive of type '<class 'object'>', but there is none available.

From felix.antoine.fortin on April 02, 2014 08:56:00

I have just pushed a modification to the creator that I would like to be reviewed and commented.

The problem: if a user create two classes with the creator.create function and both of these classes have the same name, the first created class will be overwritten and no warning will be expressed. While this can be tolerable if the creation of classes could only occur in the same module, where it would be relatively easy to find out that two classes with the same name have been created, if the classes creation is done in multiple module, chaos will ensue.

I think the global aspect of class creation with the creator has its benefits. However, I think we should warn users of potential problem with this very aspect.

See revision bf3204c3e5fc

Original issue: http://code.google.com/p/deap/issues/detail?id=36

I'm trying to run Python script that uses DEAP and SCOOP to process evaluations in parallel, however I get the following traceback.

[2015-02-12 15:24:25,021] brokerzmq (127.0.0.1:57627) DEBUG   SHUTDOWN command received.
Traceback (most recent call last):
  File "/Applications/Canopy.app/appdata/canopy-1.1.0.1371.macosx-x86_64/Canopy.app/Contents/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/Applications/Canopy.app/appdata/canopy-1.1.0.1371.macosx-x86_64/Canopy.app/Contents/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/bootstrap/__main__.py", line 302, in <module>
    b.main()
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/bootstrap/__main__.py", line 92, in main
    self.run()
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/bootstrap/__main__.py", line 290, in run
    futures_startup()
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/bootstrap/__main__.py", line 271, in futures_startup
    run_name="__main__"
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/futures.py", line 64, in _startup
    result = _controller.switch(rootFuture, *args, **kargs)
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/_control.py", line 253, in runController
    raise future.exceptionValue
IndexError: string index out of range

I've tried to run it through debugging, but I can never reach the code in _control.py because the program terminates before then with the following traceback.

[2015-02-12 15:51:56,111] scoopzmq  (127.0.0.1:51799) ERROR   An instance could not find its base reference on a worker. Ensure that your objects have their definition available in the root scope of your program.
'module' object has no attribute 'Individual'
[2015-02-12 15:51:56,111] brokerzmq (127.0.0.1:51893) DEBUG   SHUTDOWN command received.
[2015-02-12 15:51:56,113] scoopzmq  (127.0.0.1:52298) ERROR   A worker exited unexpectedly. Read the worker logs for more information. SCOOP pool will now shutdown.
Traceback (most recent call last):
  File "/Applications/Canopy.app/appdata/canopy-1.1.0.1371.macosx-x86_64/Canopy.app/Contents/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/Applications/Canopy.app/appdata/canopy-1.1.0.1371.macosx-x86_64/Canopy.app/Contents/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/bootstrap/__main__.py", line 302, in <module>
    b.main()
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/bootstrap/__main__.py", line 92, in main
    self.run()
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/bootstrap/__main__.py", line 290, in run
    futures_startup()
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/bootstrap/__main__.py", line 271, in futures_startup
    run_name="__main__"
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/futures.py", line 64, in _startup
    result = _controller.switch(rootFuture, *args, **kargs)
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/_control.py", line 207, in runController
    future = execQueue.pop()
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/_types.py", line 320, in pop
    self.updateQueue()
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/_types.py", line 343, in updateQueue
    for future in self.socket.recvFuture():
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/_comm/scoopzmq.py", line 279, in recvFuture
    received = self._recv()
  File "/Users/gvdb/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/scoop/_comm/scoopzmq.py", line 197, in _recv
    raise ReferenceBroken(e)
scoop._comm.scoopexceptions.ReferenceBroken: 'module' object has no attribute 'Individual'

I understand that this may not necessarily be an issue with DEAP, so I will also post this query to SCOOP developers for help. Any help or advice you can provide would be appreciated.

I have a use-case that may apply to others and warrant a new feature. Often, one wants to be able to reproduce the results of an evolution by supplying the same random seed. With the current implementation, this precludes running multiple instances of DEAP in parallel, since the randomness is supplied by the single instance of random.Random obtained from import random.

It would be great if there were an option to supply a user-created random.Random instance so multiple evolutions could be run in parallel without their RNGs sharing state, allowing for reproducibility.

From [email protected] on February 25, 2014 08:57:08

What steps will reproduce the problem? 1. pool = multiprocessing.Pool(4) (in main)
2. toolbox.register("map", pool.map)
3. Run the script What is the expected output? What do you see instead? Having 4 processors I would expect 4 would be used. I see 4 additional python processes but only one is used. What version of the product are you using? On what operating system? I am using v1.0.0 on windows 7 using python 2.7.6 Please provide any additional information below. None

Attachment: basic_numpy.py

Original issue: http://code.google.com/p/deap/issues/detail?id=25

Since the codebase still contains some Python 2 only syntax, when one tries to build the documentation with Python 3 without first applying 2to3 on DEAP's modules, the compilation crashes.

There are only two syntax elements still present in deap's code that are not compatible with Python 3.

• print as statement instead of a function;
• raising exception as triples instead of an object.

There are other things that are not compatible with Python 3 (for example xrange), but documentation building is only concerned with syntax.

The problem with print can be easily solved, simply add parentheses.

However, the exception raising is more problematic, since they have change how traceback can be included in Python 3. Two solutions

• ditch traceback;
• import six, a module to support codebase in both Python 2 and 3.

From [email protected] on March 01, 2013 10:14:27

First of all: The items from Wiki/Changes should be somewhere in the documentation if you rename functions and don't keep (deprecated) references to the old names. I even would suggest a porting guide if you break the API.

Besides of that: What happened with gp.mutShrink?

Original issue: http://code.google.com/p/deap/issues/detail?id=16

From felix.antoine.fortin on April 03, 2014 09:43:32

Following a question [1] regarding which selection operators can and cannot be used with eaSimple, it appears that the description of the simplest algorithm in DEAP is actually quite convoluted while missing crucial details.

Short list of problems:

• mu, lambda and k are mentioned at the very beginning, but never used afterward and no explanation is provided as what these variables represent.
• The notation for the population P(0), P(g+1), P'(g+1) from the book by Back et al. is not easy to understand without reading the reference, which is not necessarily accessible to every user.
• The selection operator registered in the toolbox has to select individuals by sampling with replacement, otherwise no selective pressure is applied. This requirement is absent from the documentation. See [1] for more details.
• The pseudocode should use the same notation used in the algorithm description.

[1] https://groups.google.com/forum/#!topic/deap-users/4nF-luBsWzA

Original issue: http://code.google.com/p/deap/issues/detail?id=37

From [email protected] on March 26, 2014 08:50:23

The hall of fame could have a mode where the best individual(s) of each generation is also saved in the independent archive.

Original issue: http://code.google.com/p/deap/issues/detail?id=33

I would suggest modifying the (examples/gp/symbreg.py) to show a new user how to limit the depth of the trees using staticLimit as having a limit is good practice given python's 90 depth limit.

The evolutionary tools clearly define each operator, (http://deap.readthedocs.org/en/latest/api/tools.html), the problem for me is learning how to glue them together.

It took quite a bit of trial and error to figure out how to correctly apply staticLimit through a decorator. I'll send a pull request in a minute.

From [email protected] on January 30, 2013 09:28:05

What steps will reproduce the problem? 1. Take an example: http://deap.gel.ulaval.ca/doc/0.9/examples/code/gp/gp_multiplexer.py 2. Added the necessary lines to work using scoop: http://deap.gel.ulaval.ca/doc/0.9/tutorials/distribution.html#scalable-concurent-operations-in-python-scoop 3. Add an ephemeral constant:
pset.addEphemeralConstant(lambda: random.random() * 10) What is the expected output? What do you see instead? It should run normally.

Error generated:

[2013-01-30 15:24:59,481] INFO SCOOP 0.6.0 final on linux2 using Python 2.7.3 (default, Oct 16 2012, 14:35:36) [GCC 4.5.4], API: 1013
[2013-01-30 15:24:59,481] INFO Deploying 2 worker(s) over 1 host(s).
[2013-01-30 15:24:59,481] INFO Worker distribution:
[2013-01-30 15:24:59,481] INFO 127.0.0.1: 1 + origin
Traceback (most recent call last):
File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/lib64/python2.7/site-packages/scoop/bootstrap/main.py", line 174, in
b.main()
File "/usr/lib64/python2.7/site-packages/scoop/bootstrap/main.py", line 40, in main
self.run()
File "/usr/lib64/python2.7/site-packages/scoop/bootstrap/main.py", line 169, in run
futures_startup()
File "/usr/lib64/python2.7/site-packages/scoop/bootstrap/main.py", line 152, in futures_startup
run_name="main"
File "/usr/lib64/python2.7/site-packages/scoop/futures.py", line 57, in _startup
result = _controller.switch(rootFuture, _args, *_kargs)
File "/usr/lib64/python2.7/site-packages/scoop/_control.py", line 200, in runController
raise future.exceptionValue
cPickle.PicklingError: Can't pickle <type 'function'>: attribute lookup builtin.function failed
[2013-01-30 15:25:00,306] INFO Root process is done.
[2013-01-30 15:25:00,307] INFO Finished cleaning spawned subprocesses.
---------------------------------------------------------------------- What version of the product are you using? On what operating system? Version: deap-0.9.0rc3
operating system: Linux 3.6.11-gentoo #1 SMP PREEMPT Thu Dec 20 20:07:07 CET 2012 x86_64 Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz GenuineIntel GNU/Linux Please provide any additional information below. I only tested with GPs.

Original issue: http://code.google.com/p/deap/issues/detail?id=15

From marc.andre.gardner on July 01, 2011 02:13:43

As revision 1cbc540a26e5 , EAP now makes use of metaclass to define the base fitness class (and ensure that the user set the weights property).

This is a good improvement, but on the other side, this change seems to prevent the pickling of base.Fitness derivated classes, which is quite annoying...
That means that EAP does not work anymore with multiprocessing nor DTM (unless you find an evolutionnary algorithm which does not make use of fitness...).

This can be asserted by :

• Running the deap/tests/test_pickle.py script :
  ...
  Ran 7 tests in 0.034s

FAILED (errors=6) # One of those failures is normal though (the lambda pickling)

• Running the mpga_onemax.py example (the current version in 0.7-dev was not updated to reflect the new EAP API, an up to date version is attached), which throws :

PicklingError: Can't pickle <class 'abc.FitnessMax'>: attribute lookup abc.FitnessMax failed

• Running anything relevant to EAP with DTM.
• Running this simple test case :

import pickle
from deap import base
from deap import creator

creator.create("FitnessMax", base.Fitness, weights=(1.0,))
creator.create("Individual", list, fitness=creator.FitnessMax)

s = creator.Individual([1,2,3,4,5])
s.fitness.values = (10,)
pickle.dumps(s)

print("Success")

Which runs fine under EAP 0.6 (e.g. print "success"), but throws an error under 0.7 :
pickle.PicklingError: Can't pickle <class 'abc.FitnessMax'>: it's not found as abc.FitnessMax

As stated in this discussion ( http://bugs.python.org/issue7689 ), this seems to be a currently unresolved CPython issue. Indeed, provide reduce or get/setstate methods to base.Fitness does not seem to work either.

Attachment: mpga_onemax.py

Original issue: http://code.google.com/p/deap/issues/detail?id=7

The question has come up a few time : why an individual inheriting from gp.PrimitiveTree cannot be pickle? See this thread for example:
https://groups.google.com/forum/#!topic/deap-users/XZv_yY89y84

We need to clarify the importance of the pickling protocol in the gp documentation in order to avoid repeating ourselves over and over.

From [email protected] on March 20, 2014 13:20:41

I recently toyed using GP and symbolic regression and realized that it would be convenient to have an infix representation of the trees readily available in deap.

For example, having a way to represent an individual as "x + y" instead of "add(x, y)" could improve the output readability of user programs using symbolic mathematics.

Here is a quick and dirty example: https://gist.github.com/soravux/9657679 I suggest the getInfix() function could be available as an helper function in deap. This is not necessarily the typical usage of GP (Trees that are not symbolic mathematic based won't benefit from this), but can be useful enough to warrant its inclusion.

The problem with the current implementation is that it requires a dictionary to map function names to their symbols. It should be left to the user to provide an str() to the primitives. This would then be used by replacing x.name by str(x). Furthermore, primitives with a real arity different from its reported arity (ie. the shifts in my example which have a second operand forced to 1 and then reported as arity 1) should be easily overridable to be represented correctly.

I am willing to provide the code for this feature.

What do you think?

Original issue: http://code.google.com/p/deap/issues/detail?id=31

From felix.antoine.fortin on March 26, 2014 09:08:47

In these examples, we open a file but we forgot to close it.

• gp/spambase.py line 33
• gp/ant.py line 186
• ga/knn.py line 76
• ga/evoknn.py line 35

We should use you the with statement.

Original issue: http://code.google.com/p/deap/issues/detail?id=34

From felix.antoine.fortin on May 06, 2013 10:21:17

I have been working recently on better way to compile statistics and evolution informations in a single object.

This new object, temporarily named "Database", can be used to compute predefined statistics like the Statistics object, but can also log arbitrary information like the generation, the deme index, or anything worthy of noting during the evolution. The object is a list of dictionary.

The statistics computation requires Numpy. It is simpler to use Numpy than implement our own statistic functions and I think it is correct to assume that anyone doing scientific computation have installed Numpy.

The prototype is available at : https://gist.github.com/cmd-ntrf/5525264 When reviewing this, please focus on the following aspects :

1. The attribute selection interface. The class provides a select method that allows to retrieve one or multiple attributes in each dictionary of the list. The method either return a single list or a list of lists. It could be interesting to provide an interface to filter results through for example a predicate function.
2. The class implements str to display its content. However, it has the disadvantage of reprinting the whole object every time it is called. I have implemented an Interface class that uses curses to refresh the screen, but it gets slower has the evolution progresses. We need to think of a way to print the object without having to reprint what has already been printed.
3. The object can use a key function. The result of this function is used to compute the statistics. Would it be useful to have multiple keys? Would it be more useful to provide a key for each statistical function instead of a global one ?
4. What do you think of using curses as an interface for DEAP? It could be used to provide the user a mechanism to interrupt the evolution by pressing a key and eventually restart it.
5. Is there a function or a workflow that used to work with the Statistics or the EvolutionLogger that could be not be reproduced with this class?

Original issue: http://code.google.com/p/deap/issues/detail?id=19

I notice the beautiful video of the algorithm in action, plotted with matplotlib. I can not find this code. Anybody have shared this?
https://www.youtube.com/v/f3MW2krobpo?version=3&feature=player_detailpage

From felix.antoine.fortin on January 23, 2013 16:34:11

It was identified that when adding two different primitives with the same name in a gp.PrimitiveSet and gp.PrimitiveSetTyped with the addPrimitive method, the primitive associated to name the name in the context dictionary corresponds to the last entry only.

The problem is that deap currently uses the name attribute of the primitive to identify their name, but primitives may come from different modules, be different object and still have the same value for this attribute.

The solution we propose is to allow the user to rename the primitive with an optional argument name in addPrimitive and make sure that all primitive names are uniques. When adding a primitive with a name already present, an assertion occurs.

The included patch fixes this issue by applying the solution aforementioned.

Attachment: name_clash.patch

Original issue: http://code.google.com/p/deap/issues/detail?id=14

@johnmee reports the following on gitter:

the docs say to put the pset that calls all the others as the last in the list of psets, but I'm finding it only works when I put the main pset first. Is the documentation backward, or am I? It does get reversed in the code...

The documentation is backward.

From [email protected] on July 19, 2013 23:12:38

What steps will reproduce the problem? 1. Ran example from the documentation site: One Max Problem: Short Version¶
2. Python returned error: AttributeError: 'module' object has no attribute 'mean' 3. What is the expected output? What do you see instead? Compile Error:

File "F:\Eclipse Projects\ga\ga.py", line 97, in main
stats.register("avg", tools.mean)
AttributeError: 'module' object has no attribute 'mean' What version of the product are you using? On what operating system? version 1.0rc Please provide any additional information below. It seems like there does not exist a mean property in the tools library.

Original issue: http://code.google.com/p/deap/issues/detail?id=21

Hi!
i want to run this example (spambase.py) in DEAP 1.0.1, but it show me the next kind of error:
... am i doing something wrong?

Regrets

From [email protected] on March 07, 2014 01:18:40

Hi

I am trying to work through the example gp_symbreg.py but i had the following problems:

1. line 41: pset.addEphemeralConstant(lambda: random.randint(-1,1))
   addEphemeralConstant is supposed to take in 3 arguments based on the documentation, so it seems to be missing a name?
2. Traceback (most recent call last):
   File "gp_symbreg.py", line 51, in
   toolbox.register("lambdify", gp.lambdify, pset=pset)
   AttributeError: 'module' object has no attribute 'lambdify

Thank you.

Original issue: http://code.google.com/p/deap/issues/detail?id=30

From felix.antoine.fortin on March 26, 2014 09:33:46

After looking up the other popular frameworks, it appears that everyone returns 1 when the denominator of the protected division is 0. I am not sure where I originally got the idea to return 0… Even in Koza original book, the list implementation of the protected division returns 1 when the denominator is zero.

(defun % (numerator denominator)
"The Protected Division Function"
(values (if (= 0 denominator) 1 (/ numerator denominator)))
)

This is also recommended in A Field Guide to Genetic Programming by Poli and Landgon p.22 footnote 1
"The decision to return the value 1 provides the GP system with a simple way to
generate the constant 1, via an expression of the form (% x x). This combined with a
similar mechanism for generating 0 via (- x x) ensures that GP can easily construct
these two important constants."

To avoid confusion, we should probably rename the function from safeDiv to protDiv or divProtected.

Original issue: http://code.google.com/p/deap/issues/detail?id=35

From [email protected] on July 14, 2013 12:08:48

Allows printing the fitness of a not yet evaluated individual. To be used on didactic examples or debugging.

Avoids this error returning a legal value on repr:

import deap
import deap.creator
import deap.base
deap.creator.create("Fitness", deap.base.Fitness, weights=(-1.0, ) )
deap.creator.Fitness
<class 'deap.creator.Fitness'>
fit = deap.creator.Fitness()
fit
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/deap/base.py", line 243, in repr
return "%s.%s(%r)" % (module, name, self.values)
File "/usr/local/lib/python2.7/dist-packages/deap/base.py", line 179, in getValues
return tuple(map(truediv, self.wvalues, self.weights))
TypeError: unsupported operand type(s) for /: 'NoneType' and 'float'

Attachment: base.py.patch

Original issue: http://code.google.com/p/deap/issues/detail?id=20

From [email protected] on April 22, 2010 19:42:50

A proposition for a naming convention.

For variables and attributes, lowercase is used (lower_case may be prefered for long or
uncommon names).

For function names camelCase is used. The prefix (first word) is lower case (and abbreviated if
possible and not the only word), it delimits broader sections, crossover, mutation, algrithms, etc.

List of proposed prefixes
Crossover = cx
Mutation = mut
Selection = sel
Migration = mig
Evolutionary Algorithm = ea
Evolution Strategy = es
Genetic Algorithm = ga
Genetic Programming

For Classes, CapitalizedWords is used. After the review, I'll merge this branch into: /trunk

Original issue: http://code.google.com/p/deap/issues/detail?id=1

Since object in the tree are never copied, we cannot regenerate the ephemeral values by simply calling a function, as it will change the value for every individual who shares that ephemeral. Instead, we need to create a complete new object.

We need to remove the regen method and create a new ephemeral in the mutEphemeral operator.

From [email protected] on January 18, 2013 10:17:47

Below is the patch text generated by mercurial. I am hoping this was the correct avenue to submit a change.

# HG changeset patch
# User Jason Zutty <[email protected]>
# Date 1358521911 18000
# Branch dev
# Node ID 72e303c713bc7dd83b1286a7109e5881b5041b97
# Parent  2bb81fcc9b7bf352e60d31814f7910152641f1d5
Added a method polymorph to class PrimitiveSetTyped.  Polymorph allows the terminals from one type to be used wherever terminals of another type are permitted.

diff -r 2bb81fcc9b7b -r 72e303c713bc deap/gp.py
--- a/deap/gp.py    Mon Jan 14 11:08:22 2013 -0500
+++ b/deap/gp.py    Fri Jan 18 10:11:51 2013 -0500
@@ -304,6 +304,12 @@
         """
         return self.terms_count / float(self.terms_count + self.prims_count)

+    def polymorph(self,supertype,subtype):
+   """Allows the use of all terminals of a subtype,
+   wherever the use of the supertype is permitted
+   """
+   self.terminals[supertype] = self.terminals[supertype] + self.terminals[subtype]
+
 class PrimitiveSet(PrimitiveSetTyped):
     """Class same as :class:`~deap.gp.PrimitiveSetTyped`, except there is no 
     definition of type.

Original issue: http://code.google.com/p/deap/issues/detail?id=12

From [email protected] on February 28, 2014 16:05:20

I took the example code, replaced the evaluate to use zdt4, and uncommented:

Functions zdt4 has bounds x1 = [0, 1], xn = [-5, 5], with n = 2, ..., 10

BOUND_LOW, BOUND_UP = [0.0] + [-5.0]_9, [1.0] + [5.0]_9

The problem in cxSimulatedBinaryBounded is that the 'low' and 'up' arguments can be iterable(lists) but are not iterators. This doesn't show up in the original example (zdt1) because the bounds are made into iterators with 'repeat'. I fixed this by adding some else statements:
size = min(len(ind1), len(ind2))
if not isinstance(low, Iterable):
low = repeat(low, size)
else:
low = iter(low)
if not isinstance(up, Iterable):
up = repeat(up, size)
else:
up = iter(up) What steps will reproduce the problem? 1. Run python 3.3 on the attached file, with deap 1.0.0 installed What is the expected output? What do you see instead? Expect a normal optimization, get:
python nsga2_zdt4.py
gen evals std min avg max
0 100 [ 0.2886756 30.82218893] [ 1.63319464e-02 8.39760854e+01] [ 0.48056234 154.56092183] [ 0.99361487 229.82580722]
Traceback (most recent call last):
File "nsga2_zdt4.py", line 126, in
pop, stats = main()
File "nsga2_zdt4.py", line 101, in main
toolbox.mate(ind1, ind2)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/deap/tools/crossover.py", line 307, in cxSimulatedBinaryBounded
xl = next(low)
TypeError: 'list' object is not an iterator What version of the product are you using? On what operating system? deap 1.0.0 on OSX 10.9 Please provide any additional information below.

Attachment: nsga2_zdt4.py

Original issue: http://code.google.com/p/deap/issues/detail?id=26

Our unit tests have been neglected and probably cover only a small portion of the code.

I would like us to try Hypothesis to test the different functions:
https://hypothesis.readthedocs.org/en/latest/

As for Travis, we should probably add the automatic execution of some of the examples as an integration test. We have deap-speed, but it does not react when someone is making a pull request.