divio / aldryn-search Goto Github PK
View Code? Open in Web Editor NEWHaystack 2.0 search index for django CMS
License: Other
Haystack 2.0 search index for django CMS
License: Other
I tried to run aldryn-jobs with aldryn-search using the whoosh SearchEngine Backend but aldryn-jobs cannot get indexed correctly:
./manage.py rebuild_index
results in ValueError: language_code can't be null
from here: https://github.com/edoburu/django-parler/blob/master/parler/utils/conf.py#L111
Temporary work around: set ALDRYN_JOBS_SEARCH
to False
in settings.py
Some plugins rely on context variables set by their parent's render()
method, because of how plugins are rendered here, the context is always new instead of being shared by plugins, thus these plugins fail to render correctly.
if haystack is used in conjunction with Elasticsearch, the shown configuration settings
HAYSTACK_CONNECTIONS = {
'en': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': 'http://localhost:9200/',
'INDEX_NAME': 'haystack-en',
},
'fr': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': 'http://localhost:9200/',
'INDEX_NAME': 'haystack-fr',
},
}
will give a very weird error:
File "lib/python2.7/site-packages/haystack/__init__.py", line 6, in <module>
from haystack import signals
ImportError: cannot import name signals
Instead declare one of the search-indices as default, say
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch_backend.ElasticsearchSearchEngine',
'URL': 'http://localhost:9200/',
'INDEX_NAME': 'haystack-en',
},
...
}
aldryn-search 0.4.0 is out on PyPi, but the source here hasn't been updated. Has the official repo moved? Is Aldryn no longer publishing the source? On the PyPi page a non-existent repo is listed instead of this one.
It seems like this issue is the same as: django-parler/django-parler#90. However, it is still plaguing aldryn-search. I am using the most current version of Parler and have setting the settings.py with default to 'en' , my only solution right now was to set all internationalization settings to False.
It seems @yakky has a solution for his djangoCMS-blog on this commit:
https://github.com/nephila/djangocms-blog/pull/303/commits
I am wondering is it possible to incorporate @yakky's fix into aldryn search?
I don't think there is a way to execute manage.py
commands on the Aldryn life server... So manage.py update_index
is out of question. (?)
Hi, when I try a fresh install of aldryn-search with pip I got the following:
Could not find a version that satisfies the requirement django-mptt==0.5.2,==0.6,==0.6.1 (from django-cms>=2.4->aldryn-search) (from versions: 0.1, 0.2, 0.2.1, 0.3.0, 0.3.1, 0.4.0, 0.4.1, 0.4.2, 0.5.0, 0.5.1, 0.5.2, 0.5.3, 0.5.4, 0.5.5, 0.6.0, 0.6.1)
No distributions matching the version for django-mptt==0.5.2,==0.6,==0.6.1 (from django-cms>=2.4->aldryn-search)
I think your requeriments file is not compatible with the PEP440. Thanks.
If yes, how do we figure out which pages contain which static placeholders?
Just been looking over the code as I'm updating a fork of the app & in utils.py
it looks to me as though ParserError
is undefined so must surely cause an exception;
https://github.com/aldryn/aldryn-search/blob/master/aldryn_search/utils.py#L147
I'm using aldryn_search.signal_processor.RealTimeSignalProcessor.
When moved a page and search, result.url is not updated. because a index is updated when only publishing or unpublishing.
{% for result in page_obj.object_list %}
{{ result.url }} {# result.url is broken. 404 Error occured. #}
And, change a page's slug(or overwrite_url) is need publish, but descendants's url are not changed.
I tried implement update all descendant's indexes when a page moved, but the process is too slowly. Also no effect in case of slug changed.
I think that there is no other way than access page object, as described below... Any ideas?
load_all = True
{{ result.object.page.get_absolute_url }}
When indexing aldryn-snippet which is only html comment (for example '' ) indexing fails.
https://github.com/aldryn/aldryn-search/blob/master/aldryn_search/utils.py#L130 fails when value has only html commnt.
I have set up aldryn-search with whoosh and I discovered that the search results will show also results from sites with login_required
, revealing private data. How can I prevent aldryn_search to show results from pages with login_required if a user is not logged in?
Kind regards,
Didier
from aldryn_search.views import AldrynSearchView
File "/home/fellu/code/own/asm_cms/env/lib/python3.3/site-packages/aldryn_search/views.py", line 9, in
from aldryn_common.paginator import DiggPaginator
File "/home/fellu/code/own/asm_cms/env/lib/python3.3/site-packages/aldryn_common/paginator.py", line 61
except InvalidPage, e:
^
SyntaxError: invalid syntax
would be nice!
Hi,
I am facing an issue when creating blog posts / publishing pages from the page listing in another language. The basic problem is, that get_language()
returns the wrong value e.g. for this link
http://localhost:8000/en/admin/cms/page/13/de/publish/?redirect_language=en&redirect_page_id=5
. The same applies for blog posts.
To clarify / how to reproduce:
Expected:
Actual:
I tried to workaround that by inspecting the instance in a custom implementation of the router, unfortunately there is not unified way to do so. The Title model for instance has a language
attribute whilst an Article has a language_code
. The only solution for now that comes to my mind would be to store all entries in one index and query with a language filter...
Any recommendations / help regarding this are greatly appreciated.
Regards
Jakob
I'm using this plugin to add search to a client site. It successfully searches CMS Pages, Aldryn Blog Posts, and my custom Location "pages". The Blog output looks great - links to the blog post and shows the Lead In.
However, with CMS Pages, the {{ result.object.get_absolute_url }}
as noted in the Haystack documentation, doesn't produce any value, when trying to link to the page. Also how can we get a description for the page? I assume it pulls the Meta Description on the page? I try using {{ result.object.body }}
like in the documentation but it's nothing.
I've just managed to set up aldryn_search and everything is working as expected. The only thing that currently bothers me is, that one of my websites has a lot of redirects which stem from relocating the relevant pages from /child/grandchild/page to /child/page.
Now the search index contains duplicate results, which both lead to the same page. I'm not sure whether I simply didn't find the proper option, or whether it doesn't exist. If this feature doesn't exist yet, I will see if I can implement it myself.
I receive the following warning every time I start the server:
site-packages/aldryn_search/utils.py:16: RemovedInDjango19Warning:
django.utils.importlib will be removed in Django 1.9.
from django.utils.importlib import import_module
Using:
Hi, users are unable to run aldryn-search due to dependency conflict with django package.
As shown in the following full dependency graph of aldryn-search, django-appconf requires django,while django-cms requires django>=1.11,<2.2.
According to pip’s “first found wins” installation strategy, django 2.2.4 is the actually installed version.
However, django 2.2.4 does not satisfy django>=1.11,<2.2.
aldryn-search-master
| +-aldryn-common(version range:>=1.0.2)
| | +-aldryn-boilerplates(version range:*)
| | | +-django(version range:>=1.11)
| | | +-django-appconf(version range:*)
| | | | +-django(version range:*)
| | | | +-six(version range:*)
| | | +-yurl(version range:>=0.13)
| | +-django-sortedm2m(version range:*)
| | +-six(version range:*)
| +-django-appconf(version range:*)
| | +-django(version range:*)
| | +-six(version range:*)
| +-django-cms(version range:>=3.4.5)
| | +-django(version range:>=1.11,<2.2)
| | +-django-classy-tags(version range:>=0.7.2)
| | | +-django(version range:>=1.11)
| | +-django-formtools(version range:>=2.1)
| | | +-django(version range:>=1.8)
| | +-django-sekizai(version range:>=0.7)
| | | +-django(version range:>=1.11)
| | | +-django-classy-tags(version range:>=0.3.1)
| | | | +-django(version range:>=1.11)
| | +-django-treebeard(version range:>=4.3)
| | +-djangocms-admin-style(version range:>=1.2)
| +-django-haystack(version range:>=2.0.0)
| | +-django(version range:>=1.11)
| +-django-spurl(version range:*)
| | +-six(version range:*)
| | +-urlobject(version range:>=2.4.0)
| +-django-standard-form(version range:*)
| | +-django(version range:>=1.3)
| | +-django-classy-tags(version range:>=0.3.3)
| | | +-django(version range:>=1.11)
| +-lxml(version range:*)
| +-setuptools(version range:*)
Thanks for your help.
Best,
Neolith
Setting this up for my latest project I get an error
from appconf import AppConf
ImportError: No module named appconf
django-appconf wasn't installed by pip for this package or it's dependencies so probably should be added to setup.py. I'm not sure if there are any specific version requirements, so I've not forked and made a pull request in this instance.
I have installed Haystack, whoosh and aldryn-search, and using django-cms 3.0
I have the following in the settings file:
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'core.haystack_backends.MultilingualWhooshEngine',
'PATH': os.path.join(PROJECT_PATH, 'whoosh_index', 'en'),
},
'default_zh-cn': {
'ENGINE': 'core.haystack_backends.MultilingualWhooshEngine',
'PATH': os.path.join(PROJECT_PATH, 'whoosh_index', 'zh-cn'),
},
}
HAYSTACK_ROUTERS = ['aldryn_search.router.LanguageRouter']
ALDRYN_SEARCH_LANGUAGE_FROM_ALIAS = lambda alias: alias.split('_')[-1]
The backend is based on this article: http://anthony-tresontani.github.io/Django/2012/09/20/multilingual-search/
The search view loads in the default language but when I go to the alternative language's url, a PageDoesNotExist error is thrown. What am I doing wrong?
Edit: Seems like I had a placeholder in the template base I was using that caused the page to crash. Removing the placeholder solved the issue.
After changing ALDRYN_SEARCH_PAGINATION = None
as suggested in the docs, no results were returned for valid queries.
The culprit is in search_items.html
, which required {% for result in page_obj.object_list %}
to become {% for result in object_list %}
.
Apologies if this has been addressed before, or if my thinking is off-base:
I have a plugin which returns a link to itself on a Page, e.g.:
def get_absolute_url(self):
return f"{self.placeholder.page.get_absolute_url(language=self.language, fallback=False)}#{self.slug}"
It would be nice if the search results allowed for the plugin to override the Title's url, as otherwise the user must click the link, and then scroll/search to again find the plugin's text somewhere on the page.
I think it could be accomplished by allowing the plugin to override the Title's get_url()
here:
https://github.com/aldryn/aldryn-search/blob/1f8c68aa6d7283d940e77d226bc9b5b9a17f2672/aldryn_search/search_indexes.py#L37-L38
Thoughts?
Having this rendered by my plugin:
<h1>The Title</h1>What
The word "The" and "TitleWhat" ends in my index. Maybe think about using strip_tags from django?
http://stackoverflow.com/questions/12824899/strip-tags-replace-tags-by-space-rather-than-deleting-them Everybody suggests regexes, but maybe lxml can do it?
Sorry if this has been already reported. I cannot find aldryn search on the Aldryn control panel.
Thank you.
Hello. In my project without this editing (https://github.com/aldryn/aldryn-search/blob/master/aldryn_search/helpers.py#L36) the content in the cmsplugin-filer is not indexed
def get_cleaned_bits(data):
decoded = force_unicode(data)
stripped = strip_tags(decoded)
return list(smart_split(stripped))
The README links to old cms-search docs
I would like to start off by stating that having the installation instructions under docs instead of the readme is really inconvenient. I did not even know it existed and I wasted a lot of hours trying to get this set up because there was no explanation in the readme or even a statement to read the items in the docs folder. Can I suggest a change to that.
I'm trying to do the last step - to rebuild_index. I have my server running in one tab and a new tab open and I try to run that and get a 403 forbidden error. I cannot find good documentation on how to do this.
Also at the end of the error message it does say:
Indexing 0 articles
Indexing 0 People
Why is it not trying to index the CMS pages? I shouldn't have to, but I did include the following in my settings.py (even though it defaults to true).
ALDRYN_SEARCH_CMS_PAGE = True
Is it possible? For now I only have
All documents removed.
Indexing 92 titles
And so the index only contains cms.title objects.
How is it possible to index the djangocms_text_ckeditor plugin instances. I suspect it it supported in this app?
I've extended aldryns TitleIndex and would like to use my own. However, haystack always complains like
django.core.exceptions.ImproperlyConfigured: Model '<class 'cms.models.titlemodels.Title'>' has more than one 'SearchIndex`` handling it.
Trying to configre my HAYSTACK_CONNECTIONS with the following:
{
'EXCLUDED_INDEXES': ['aldryn_search.search_indexes.TitleIndex', ]
}
But this only works for my own index, the above does not work? Any experiences on this behaviour?
This requires Django 1.8: from django.template import Engine
(https://github.com/aldryn/aldryn-search/blob/master/aldryn_search/helpers.py#L3, introduced in df389ce)
I guess the Django version checked for in the new code needs to be change?
We need to allow that each object in search results be validate against permissions, mostly user permissions. This need to be done when searching and creating the "queryset", or when iterating over SearchQuerySet
(since we make count and slice work well because consistency and pagination).
It can't be done when indexing because permissions can change between time it was indexed and searched.
The SearchResult
instance has the searchindex
attribute, that is the SearchIndex
class used to index the object, so in last case we can add some 'check_permission' method in it.
In my opinion a better idea is to customize SearchBackend.search
to check permissions for each result before return it. This method as haystack documentation explains returns one "dictionary that should contain the keys ‘results’ and ‘hits’. The ‘results’ value should be an iterable of populated SearchResult objects." and this looks a nice place to apply permission checking and ignore some result objects.
However we must discuss how apply that improvement on haystack in aldryn-search.
Greetings!
I happen to notice that there was recently a PR that was merged into into the master involving excluding plugins that aren't compatible such as DjangoCMS Video and Audio.
I was wondering if or when will you make those release public or is it already there?
Hi,
I want to make the search form available on my menu bar, but i cant see how i make this app globally available.
Maybe this is a dumb question.
Hey guys, I keep getting this error. I can interact with elasticsearch through python shell without a problem, and update indexes. I can also connect to elasticsearch service through http. I think somehow aldryn_search is involved, which is why I created this issue here.
Here is my settings.py just in case http://pastebin.com/BbxDen35
(env)root@vagrant-ubuntu-trusty-64:/var/outermeasure/app# python manage.py update_index
/var/outermeasure/app/env/local/lib/python2.7/site-packages/app_data/fields.py:2: DeprecationWarning: django.utils.simplejson is deprecated; use json instead.
from django.utils import simplejson as json
No handlers could be found for logger "elasticsearch"
Indexing 3 titles
ERROR:root:Error updating cms using default
Traceback (most recent call last):
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 221, in handle_label
self.update_backend(label, using)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 266, in update_backend
do_update(backend, index, qs, start, end, total, self.verbosity)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 89, in do_update
backend.update(index, current_qs)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/haystack/backends/elasticsearch_backend.py", line 161, in update
prepped_data = index.full_prepare(obj)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/haystack/indexes.py", line 204, in full_prepare
self.prepared_data = self.prepare(obj)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/aldryn_search/base.py", line 40, in prepare
request = self.get_request_instance(obj, current_language)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/aldryn_search/base.py", line 47, in get_request_instance
return get_request(language)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/aldryn_search/helpers.py", line 54, in get_request
request_factory = RequestFactory(HTTP_HOST=settings.ALLOWED_HOSTS[0])
IndexError: list index out of range
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
utility.execute()
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 242, in run_from_argv
self.execute(*args, **options.__dict__)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 285, in execute
output = self.handle(*args, **options)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 195, in handle
return super(Command, self).handle(*items, **options)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/django/core/management/base.py", line 385, in handle
label_output = self.handle_label(label, **options)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 221, in handle_label
self.update_backend(label, using)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 266, in update_backend
do_update(backend, index, qs, start, end, total, self.verbosity)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 89, in do_update
backend.update(index, current_qs)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/haystack/backends/elasticsearch_backend.py", line 161, in update
prepped_data = index.full_prepare(obj)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/haystack/indexes.py", line 204, in full_prepare
self.prepared_data = self.prepare(obj)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/aldryn_search/base.py", line 40, in prepare
request = self.get_request_instance(obj, current_language)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/aldryn_search/base.py", line 47, in get_request_instance
return get_request(language)
File "/var/outermeasure/app/env/local/lib/python2.7/site-packages/aldryn_search/helpers.py", line 54, in get_request
request_factory = RequestFactory(HTTP_HOST=settings.ALLOWED_HOSTS[0])
IndexError: list index out of range
I am posting this here as it's essential for aldryn-search for this to work.
The SimpleEngine
backend doesn't return any search results (while haystack.backends.whoosh_backend.WhooshEngine
does, exact same settings).
I tried adding the apphook to a page and using the default search form. Result: "No results found."
On the shell I am able to see search results (based on http://django-haystack.readthedocs.io/en/v2.4.1/debugging.html#no-results-found-on-the-web-page) however running
from haystack.query import SearchQuerySet
sqs = SearchQuerySet().all()
for result in sqs:
print (sqs.text)
shows that no result has a property text.
Settings by AppConf will not be populated in some cases running tests with pytest...
The tests failed because of something like:
AttributeError: 'Settings' object has no attribute 'ALDRYN_SEARCH_DEFAULT_LANGUAGE'
AttributeError: 'Settings' object has no attribute 'ALDRYN_SEARCH_INDEX_BASE_CLASS'
Normaly #85 address this and fix it. But in my cases not. Don't know why... Maybe other people has the same problems and may give some feedback... Think this is not a real bug with aldryn-search, so i will close this issues, after creation ;)
My currently solution is a Pytest fixture that will invoked automatically and set the settings, see: https://gist.github.com/jedie/c119e73d8e7dad48183f9622b430373f
See also: django-compressor/django-appconf#30
I use djangocms-blog and the search-function that comes with it (more or less).
When I set
HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'
I get an AttributeError in aldryn_search/base.py line 37 when I want to add a new blog-post:
AttributeError: 'PostIndex' object has no attribute '_backend_alias'
A simple solution would be to check if _backend_alias is set and set it to None if its not (or DEFAULT_ALIAS from haystack).
from haystack.constants import DEFAULT_ALIAS
...
def prepare(self, obj):
if not hasattr(self, '_backend_alias'):
self._backend_alias = DEFAULT_ALIAS # or set to None
...
Would this be possible to implement in an upcoming version or do you think the error is somewhere completely else?
Here is a part of the traceback I get:
File "/usr/local/lib/python3.6/site-packages/djangocms_blog/models.py", line 255, in save
super(Post, self).save(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/parler/models.py", line 657, in save
super(TranslatableModelMixin, self).save(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/django/db/models/base.py", line 796, in save
force_update=force_update, update_fields=update_fields)
File "/usr/local/lib/python3.6/site-packages/django/db/models/base.py", line 833, in save_base
update_fields=update_fields, raw=raw, using=using)
File "/usr/local/lib/python3.6/site-packages/django/dispatch/dispatcher.py", line 191, in send
response = receiver(signal=self, sender=sender, **named)
File "/usr/local/lib/python3.6/site-packages/haystack/signals.py", line 52, in handle_save
index.update_object(instance, using=using)
File "/usr/local/lib/python3.6/site-packages/haystack/indexes.py", line 284, in update_object
backend.update(self, [instance])
File "/usr/local/lib/python3.6/site-packages/haystack/backends/elasticsearch_backend.py", line 168, in update
prepped_data = index.full_prepare(obj)
File "/usr/local/lib/python3.6/site-packages/haystack/indexes.py", line 208, in full_prepare
self.prepared_data = self.prepare(obj)
File "/usr/local/lib/python3.6/site-packages/aldryn_search/base.py", line 37, in prepare
current_language = self.get_current_language(using=self._backend_alias, obj=obj)
AttributeError: 'PostIndex' object has no attribute '_backend_alias'
I'm using HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor'
and anticipate that searching for some text is successful within a few seconds of publishing a page that has the text.
manage.py update_index
, but that isn't desired.What is the expected behavior when changing text plugin contents with real time updates?
Django==1.7.10
django-cms==3.1.4
aldryn-search==0.2.10
Python 3.4.3
etc.
TIA!
The code at this line should check if ALLOWED_HOSTS is actually populated:
https://github.com/aldryn/aldryn-search/blob/master/aldryn_search/helpers.py#L64
I've got an issue updating indexes coming from TitleIndex
where the page body is not being decoded.
Traceback (most recent call last):
File "/Applications/PyCharm 2.app/Contents/helpers/pydev/pydevd.py", line 1580, in <module>
globals = debugger.run(setup['file'], None, None, is_module)
File "/Applications/PyCharm 2.app/Contents/helpers/pydev/pydevd.py", line 964, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/Users/mwalker/Sites/myproj/myproj/manage.py", line 22, in <module>
execute_from_command_line(sys.argv)
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
utility.execute()
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/django/core/management/__init__.py", line 359, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/django/core/management/base.py", line 294, in run_from_argv
self.execute(*args, **cmd_options)
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/django/core/management/base.py", line 345, in execute
output = self.handle(*args, **options)
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 214, in handle
self.update_backend(label, using)
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 257, in update_backend
commit=self.commit, max_retries=self.max_retries)
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 84, in do_update
backend.update(index, current_qs, commit=commit)
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/haystack/backends/elasticsearch_backend.py", line 190, in update
bulk(self.conn, prepped_docs, index=self.index_name, doc_type='modelresult')
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/elasticsearch/helpers/__init__.py", line 190, in bulk
for ok, item in streaming_bulk(client, actions, **kwargs):
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/elasticsearch/helpers/__init__.py", line 162, in streaming_bulk
for result in _process_bulk_chunk(client, bulk_actions, raise_on_exception, raise_on_error, **kwargs):
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/elasticsearch/helpers/__init__.py", line 87, in _process_bulk_chunk
resp = client.bulk('\n'.join(bulk_actions) + '\n', **kwargs)
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
return func(*args, params=params, **kwargs)
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 785, in bulk
doc_type, '_bulk'), params=params, body=self._bulk_body(body))
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/elasticsearch/transport.py", line 327, in perform_request
status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/elasticsearch/connection/http_requests.py", line 68, in perform_request
response = self.session.request(method, url, data=body, timeout=timeout or self.timeout)
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/requests/adapters.py", line 423, in send
timeout=timeout
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/Users/mwalker/Sites/myproj/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 356, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1053, in request
self._send_request(method, url, body, headers)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1093, in _send_request
self.endheaders(body)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1049, in endheaders
self._send_output(message_body)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 891, in _send_output
msg += message_body
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1815: ordinal not in range(128)
I'm using;
aldryn-search==0.3.0
Django==1.10.7
django-cms==3.4.3
django-haystack==2.6.0
elasticsearch==2.4.1
requests==2.13.0
requests-aws4auth==0.9
awsauth = AWS4Auth(TBH_AWS_ACCESS_KEY, TBH_AWS_SECRET_KEY, AWS_S3_REGION, 'es')
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.elasticsearch2_backend.Elasticsearch2SearchEngine',
'URL': 'https://search.eu-west-1.es.amazonaws.com/',
'INDEX_NAME': 'myproj_dev',
# The default 10 seconds is typically not enough!
'TIMEOUT': 30,
'KWARGS': {
'port': 443,
'http_auth': awsauth,
'use_ssl': True,
'verify_certs': True,
'connection_class': elasticsearch.RequestsHttpConnection,
}
},
}
Once the process hits urllib3 at the following code block;
# conn.request() calls httplib.*.request, not the method in
# urllib3.request. It also calls makefile (recv) on the socket.
if chunked:
conn.request_chunked(method, url, **httplib_request_kw)
else:
conn.request(method, url, **httplib_request_kw)
I can get the following data out;
>>> httplib_request_kw['body'][1815]
'�'
>>> httplib_request_kw['body'].decode('utf-8')[1815]
u'-'
Is there anything that can be done in the search indexes to avoid this?
The readme mentions that spurl
must be included in INSTALLED_APPS
in order to use the built-in
aldryn search view, but makes no mention that standard_form
must be as well, on pain of TemplateSyntaxError
!
https://github.com/aldryn/aldryn-search/blob/master/aldryn_search/utils.py#L92
model_or_string = '{0},{1}'.format([app_label, model_name])
I think there are a few things wrong here. The extra [] around app_label, model_name gives a IndexError. Also there is a ,
separating the label and name but I think it should be .
I would guess it's supposed to be:
model_or_string = '{0}.{1}'.format(app_label, model_name)
If one uses a string instead of a Model this issue is bypassed. For example this works.
class PlacesSearchView(AldrynSearchView):
models = ['places.place']
Hi there. After configuring the search, I've come across an error while doind a rebuild_index
, as follows:
TypeError: sequence item 1: expected string or Unicode, instancemethod found
Traceback (most recent call last):
File "/home/marcos/django/project.git/env/local/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 221, in handle_label
self.update_backend(label, using)
File "/home/marcos/django/project.git/env/local/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 266, in update_backend
do_update(backend, index, qs, start, end, total, self.verbosity)
File "/home/marcos/django/project.git/env/local/lib/python2.7/site-packages/haystack/management/commands/update_index.py", line 89, in do_update
backend.update(index, current_qs)
File "/home/marcos/django/project.git/env/local/lib/python2.7/site-packages/haystack/backends/whoosh_backend.py", line 189, in update
doc = index.full_prepare(obj)
File "/home/marcos/django/project.git/env/local/lib/python2.7/site-packages/haystack/indexes.py", line 204, in full_prepare
self.prepared_data = self.prepare(obj)
File "/home/marcos/django/project.git/env/local/lib/python2.7/site-packages/aldryn_search/base.py", line 42, in prepare
self.prepared_data['text'] = self.get_search_data(obj, current_language, request)
File "/home/marcos/django/project.git/env/local/lib/python2.7/site-packages/aldryn_search/search_indexes.py", line 65, in get_search_data
return clean_join(' ', text_bits)
File "/home/marcos/django/project.git/env/local/lib/python2.7/site-packages/aldryn_search/utils.py", line 32, in clean_join
return separator.join(filter(None, iterable))
HAYSTACK_CONNECTIONS = {
'default': {
'ENGINE': 'haystack.backends.whoosh_backend.WhooshEngine',
'PATH': os.path.join(PROJECT_PATH, '../../whoosh_index'),
},
}
HAYSTACK_ROUTERS = ['aldryn_search.router.LanguageRouter', ]
ALDRYN_SEARCH_LANGUAGE_FROM_ALIAS = None
ALDRYN_SEARCH_PAGINATION = 10
I'm using Django 1.5.8
After upgrading from django-cms 3.4.5 tot 3.5 the apphook is gone from the application-settings dropdown in advanced page settings
Django newer versions don't support QuerySetPaginator
and I am getting errors while reindexing. I am explicitly getting this error.
ImportError: cannot import name 'QuerySetPaginator' from 'django.core.paginator'
When I commented out the above-mentioned class QuerySetPaginator
from site-packages it works properly.
N.B: Aldryn-common is archived by authors, but why this popular library isn't updating their dependency.
when running multiple sites in one django CMS instance, results are displayed from all sites. resulting in 404s for results from other sites. I can see there is already a site_id
field on the search index, so this could be done in here https://github.com/aldryn/aldryn-search/blob/master/aldryn_search/views.py#L62 , probably without much hassle?
If wanted at all? One issue could be, if one has other indexes, without the site_id field.
Hi,
another issue I am having is that a page once published it will not be evicted from the index when unpublished again, since should_update
returns false and no delete action is issued. This could be solved by adding a published
flag on the index and filter on query time. To avoid doubling the index size for pages we could even check if instance.page.publisher_public
exists, to make sure we do not add title drafts that have no public page (meaning they are draft only). If get_published
on the index by default returns true this should be mostly backwards compatible. The exception would be a newer index implementation in for e.g. aldryn newsblog and an outdated aldryn search implementation (which then would not filter on the published flag). If you agree that this would be a good idea, I would be happy to create pull requests for this and the other aldryn repositories.
Regards
Jakob
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.