Git Product home page Git Product logo

ckan / ckan Goto Github PK

View Code? Open in Web Editor NEW
4.3K 193.0 1.9K 204.11 MB

CKAN is an open-source DMS (data management system) for powering data hubs and data portals. CKAN makes it easy to publish, share and use data. It powers catalog.data.gov, open.canada.ca/data, data.humdata.org among many other sites.

Home Page: https://ckan.org/

License: Other

Python 72.50% JavaScript 7.07% Shell 0.14% CSS 9.75% HTML 7.56% PLpgSQL 0.08% Dockerfile 0.02% Mako 0.07% Smarty 0.01% Gherkin 0.23% SCSS 2.58%
python open-data data catalog api ckan ckanext digitalpublicgoods dpg sdg16

ckan's Introduction

CKAN: The Open Source Data Portal Software

License

Documentation

Support on StackOverflow

Build Status

Coverage Status

Chat on Gitter

CKAN is the world’s leading open-source data portal platform. CKAN makes it easy to publish, share and work with data. It's a data management system that provides a powerful platform for cataloging, storing and accessing datasets with a rich front-end, full API (for both data and catalog), visualization tools and more. Read more at ckan.org.

Installation

See the CKAN Documentation for installation instructions.

Support

If you need help with CKAN or want to ask a question, use either the ckan-dev mailing list, the CKAN chat on Gitter, or the CKAN tag on Stack Overflow (try searching the Stack Overflow and ckan-dev archives for an answer to your question first).

If you've found a bug in CKAN, open a new issue on CKAN's GitHub Issues (try searching first to see if there's already an issue for your bug).

If you find a potential security vulnerability please email [email protected], rather than creating a public issue on GitHub.

Contributing to CKAN

For contributing to CKAN or its documentation, see CONTRIBUTING.

Mailing List

Subscribe to the ckan-dev mailing list to receive news about upcoming releases and future plans as well as questions and discussions about CKAN development, deployment, etc.

Community Chat

If you want to talk about CKAN development say hi to the CKAN developers and members of the CKAN community on the public CKAN chat on Gitter. Gitter is free and open-source; you can sign in with your GitHub, GitLab, or Twitter account.

The logs for the old #ckan IRC channel (2014 to 2018) can be found here: https://github.com/ckan/irc-logs.

Wiki

If you've figured out how to do something with CKAN and want to document it for others, make a new page on the CKAN wiki and tell us about it on the ckan-dev mailing list or on Gitter.

Copying and License

This material is copyright (c) 2006-2023 Open Knowledge Foundation and contributors.

It is open and licensed under the GNU Affero General Public License (AGPL) v3.0 whose full text may be found at:

http://www.fsf.org/licensing/licenses/agpl-3.0.html

ckan's People

Contributors

amercader avatar aron avatar brew avatar domoritz avatar gauravp-nec avatar gocemitevski avatar joetsoi avatar johnbywater avatar johnglover avatar johnmartin avatar jqnatividad avatar jvickery-tbs avatar kindly avatar kowh-ai avatar nigelbabu avatar pdelboca avatar pudo avatar rossjones avatar rufuspollock avatar seanh avatar sebbacon avatar smotornyuk avatar teajaymars avatar tino097 avatar tobes avatar tomecirun avatar torfsen avatar vitorbaptista avatar wardi avatar zharktas avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ckan's Issues

/apps should have 9 not 8 items per page

On the related items / apps page for ckan 2.0 it's a 3 column grid and 8 items is the default number to show. Which means that you get one less than is supposed to show (see attached screenshot)

Screen Shot 2013-01-15 at 13 42 28

Should be super simple to fix.

Exception: The CKAN and datastore database are the same.

The default values for the datastore settings in the ini file template don't work:

ckan.datastore.write_url = postgresql://ckanuser:pass@localhost/ckantest
ckan.datastore.read_url = postgresql://readonlyuser:pass@localhost/ckantest

gives:

Traceback (most recent call last):
  File "/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/nose/suite.py", line 208, in run
    self.setUp()
  File "/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/nose/suite.py", line 291, in setUp
    self.setupContext(ancestor)
  File "/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/nose/suite.py", line 300, in setupContext
    self.config.plugins.startContext(context)
  File "/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/nose/plugins/manager.py", line 99, in __call__
    return self.call(*arg, **kw)
  File "/home/seanh/.virtualenvs/ckan/local/lib/python2.7/site-packages/nose/plugins/manager.py", line 167, in simple
    result = meth(*arg, **kw)
  File "/home/seanh/Projects/ckan/ckan/ckan/ckan_nose_plugin.py", line 40, in startContext
    plugin.configure(config)
  File "/home/seanh/Projects/ckan/ckan/ckanext/datastore/plugin.py", line 59, in configure
    self._check_separate_db()
  File "/home/seanh/Projects/ckan/ckan/ckanext/datastore/plugin.py", line 134, in _check_separate_db
    raise Exception("The CKAN and datastore database are the same.")
Exception: The CKAN and datastore database are the same.

Search should AND terms not OR terms

Originally: http://trac.ckan.org/ticket/2331 (opened 2012-04-28)

Original Description

Appears current default search in CKAN ORs terms rather than ANDing them (i.e. adding more terms increasing number of items found rather than reducing it).

Not sure when this crept in or if it has been there for a long time.

Discussion

There was quite a lot of back and forth discussion between myself and @kindly :-)

From using e.g. datahub.io I cannot emphasize how much this really is a UX problem (I believe @shevski has also found this an issue). When I search and add terms I expect to get fewer results and it would also help improve ordering / relevancy as you add terms (which can be quite poor on a site like datahub with many datasets)

I note this is minor and non-urgent but it is also probably very easy to do so one day if someone has the spare 10m needed it would be great to see this go in.

Header template isn't verbose enough

Essentially the ckan/templates/header.html doesn't have enough {% block %}s within it and it make's customizing the header within extensions a pain. Let's add more descriptive blocks.

Simple search does not support pagination

If you have set ckan.simple_search = 1 in your config, pagination in the search does not work. What happens is that the results form page 1 occur on any other page as well.

Unnecessary space under failed preview

On a resource page where the preview is failing, there is a huge white box pushing the 'additional info' off the bottom of my screen. It should be immediately under the preview failure message.
where-is-my-info

Don't commit too soon on package_create and package_update

When calling package_owner_org_update a commit is fired, which triggers the notify hook. This will cause the dataset to be indexed before it is passed to extensions. We should pass a defer_commit flag on the object.

Pull request coming.

Fix related items activity streams

It looks wrong to me: {actor} created the link to related {related_type} {related_item}, should be {actor} added the {related_type} {related_item} to the dataset {dataset} e.g. seanh added the application Bike Share Map to the dataset Bikes, or if a related item is added that does not belong to any dataset (can be done via API) then just {actor} added the {related_type} {related_item}.

Also add activities for updating and delete related items.

Original bug at okfn/ckanext-pdeu#31

New activities count "disappears" when viewing dashboard page

A number of users have been confused by the way the new activities count resets itself to 0 a few seconds after loading your dashboard page. Viewing the dashboard automatically marks all your activities as old, and a second later the count refreshes to 0 to reflect this. It seems like it might be less confusing if the count simply didn't refresh (and would change to 0 after the next page load).

Update API docs

API v3 should be promoted as the default version of the API and moved to the front page of the API docs, move API v1/2 to a legacy API page. Many of CKAN 2.0's new features are only supported in API v3.

dashboard_activity_list_html 500s

It looks like commit e27b46b added an id arg to dashboard_activity_list_html() but didn't document it, now calling the function as described in the API docs 500s. Also looks like this id param is unnecessary as it should just use the id of the authorized user. Then the code that calls this function and passes an id in can be changed to not pass an id.

Also, are there any tests for this function? Apparently not or they would be failing. Need to add tests. It's just a simple wrapper function so should only need a couple of simple tests.

Datastore docs and template ini file settings don't match

For example in the doc it's writeuser and readonlyuser, but in the template ini file it's ckanuser and readonlyuser. This means when a user copy-pastes the instructions and uses the default ini file the commands will fail. There may be other discrepancies

Missing count on `/apps` page

It says "Showing items 1 - of 80 related items found", should say "Showing items 1 - 8 of 80 related items found":

Screenshot from 2012-12-21 15:28:15

Do a final sweep of all the front-end helper text/copy on CKAN 2.0

As it stands we have mixed standards for different things within the CKAN copy. I would like to suggest the following for our copy:

  • Button / pseudo-button action links text should: caps on first letter. E.g. "Create new dataset" not "Create New Dataset", "Show more tags" not "Show More Tags", etc.
  • Lists should never end in a full stop
  • Headers should be like buttons, but the context can be capitalised. E.g. "What are Organizations?" not "What are organizations?"
  • Create action links should read: "Add new group" not "Add group"
  • Form submits should read: "Update group" or "Add new group" not "Update Group" or "Add New Group"
  • Primary action buttons for a page should always have btn-primary on them. Eg: "Edit my profile" on viewing your profile page
  • ... also they should be declarative of what they are editing. E.g. "Edit my profile" not "Edit"

Also, the /about page needs updating to be something a little nicer and the footer links need pointing to the correct pages.

ckanext.datastore leaves garbage even after being unloaded

cc @kindly @domoritz

In https://github.com/okfn/ckan/blob/master/ckanext/datastore/plugin.py#L71-L96, we're wrapping the resource_show action, adding datastore_active to the data dict. As far as I could find, there's no way of undoing it afterwards. Even plugin.unload, in the teardown in the tests, don't work.

Right now, just this test https://github.com/okfn/ckan/blob/master/ckan/tests/logic/test_action.py#L746-L754 is failing:

FAIL: ckan.tests.logic.test_action.TestAction.test_26_resource_show
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/vagrant/pyenv/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/vagrant/pyenv/src/ckan/ckan/tests/logic/test_action.py", line 754, in test_26_resource_show
    assert result == resource_dict, (result, resource_dict)
AssertionError: ({u'resource_group_id': u'f11640bd-9251-4290-ad3e-a5ca196aa574', u'cache_last_updated': None, u'size_extra': u'123', u'webstore_last_updated': None, u'datastore_active': False, u'id': u'53da9e6b-d043-471b-9939-072bf5dbe580', u'size': None, u'state': u'active', u'hash': u'abc123', u'description': u'Full text. Needs escaping: " Umlaut: \xfc', u'format': u'plain text', u'alt_url': u'alt123', u'tracking_summary': {u'total': 0, u'recent': 0}, u'mimetype_inner': None, u'mimetype': None, u'cache_url': None, u'name': None, u'created': u'2013-01-28T18:08:34.621101', u'url': u'http://www.annakarenina.com/download/x=1&y=2', u'webstore_url': None, u'last_modified': None, u'position': 0, u'revision_id': u'e5ac7c17-8a3f-42db-a382-888078c7dead', u'resource_type': None}, {'resource_group_id': u'f11640bd-9251-4290-ad3e-a5ca196aa574', 'hash': u'abc123', 'description': u'Full text. Needs escaping: " Umlaut: \xfc', 'format': u'plain text', u'alt_url': u'alt123', u'size_extra': u'123', 'mimetype_inner': None, 'webstore_last_updated': None, 'id': u'53da9e6b-d043-471b-9939-072bf5dbe580', 'size': None, 'mimetype': None, 'cache_url': None, 'name': None, 'created': '2013-01-28T18:08:34.621101', 'url': u'http://www.annakarenina.com/download/x=1&y=2', 'cache_last_updated': None, 'tracking_summary': {'total': 0, 'recent': 0}, 'webstore_url': None, 'state': u'active', 'last_modified': None, 'position': 0, 'revision_id': u'e5ac7c17-8a3f-42db-a382-888078c7dead', 'resource_type': None})

What's the best way to fix it?

Fill in format field when user uploads the data

When adding resources to datasets, when the user uploads the data, we can tell the format (and it will be filled in once we exit the wizard) - but this should get filled in the form once upload is finished to avoid user adding an incorrect format

Mark IGenshiStreamFilter as legacy

and make sure this shows up in the plugin interfaces docs

also any other plugin interfaces that should be marked legacy or deprecated

Review use of help text for users/publishers in CKAN 2.0

We need helper text for instance owners at the right points (i.e. when they're adding orgs or groups or members) - and this needs to be different to any help text that displays in the interface for end users/consumers.

I suggest that we have limited help text on the public interface. E.g. I don't see how the sidebar here: http://organizations.staging.ckanhosted.com/organization is helpful. It should be departments & their descriptions (e.g. office of national statistics, department of help) and a header that makes it clear you can view the datasets these departments publish.

Footer links don't go anywhere

The links in the site footer (Code of Conduct, etc.) don't go anywhere. This seems to be hard-coded:

<li><a href="#">{{ _('Terms and Conditions') }}</a></li>
<li><a href="#">{{ _('Accessibility') }}</a></li>
<li><a href="#">{{ _('Code of conduct') }}</a></li>
<li><a href="#">{{ _('Moderation policy') }}</a></li>
<li><a href="#">{{ _('FAQ') }}</a></li>
<li><a href="#">{{ _('Privacy') }}</a></li>
<li><a href="#">{{ _('Contact us') }}</a></li>
<li><a href="#">{{ _('Consultation') }}</a></li>

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.