socialplanning / opencore Goto Github PK
View Code? Open in Web Editor NEWSoftware that drives CoActivate.org
Home Page: http://coactivate.org/
License: GNU General Public License v3.0
Software that drives CoActivate.org
Home Page: http://coactivate.org/
License: GNU General Public License v3.0
Many people on the /manage_membership page of a mailing list are displayed by email, even though the email-to-user-account function should be able to find their accounts.
Somehow we sometimes have multiple user accounts with the same email address.
Need to investigate.
They're already there on export, but not in a very machine-friendly format -- they just exist in an <h1>
on the baked export.
Should add it to the JSON map file as well. (I think I have a JSON somewhere?)
In some test runs of Listen export->reimport, I noticed what look like some content encoding issues -- blobby-question-mark entities appearing in the reimported archives where they were not present in the original.
Need to reproduce this bug and fix it.
Right now I'm just hardcoding all projects to have listen, and nothing else, on reimport.
And I'm currently setting the homepage for all projects to /summary
.
Need to pull these preferences from the settings.ini instead.
I don't know offhand if it's possible to preserve original creation dates and creators when reimporting content from an export. If we can't, it's not a blocker, but it would be a shame to lose that data, so it's worth investigating.
This pertains to all major content types -- members, projects, mailing lists, wiki pages -- I assume the answer (and technique, if there is one) will be the same for all content types.
Write a validation/migration script to run through the whole site database to find & correct any other manifestations of bug #43
Wiki attachments are currently losing some information in export:
These need to be somehow added to the export, and then also factored into the reimport code.
A simple way to do this would be adding a microformatted index.html
page to each page subdirectory that has attachments, e.g.:
<ul>
<li><a href="attachment-1.pdf">Title For First Attachment</a> uploaded on <time pubdate datetime="2005-11-13T09:00Z"></time> by <span class="author">ejucovy</span></li>
...
</ul>
We could then just parse that on reimport.
We currently support export of projects' contents, but do not export the metadata about the project itself.
This needs to include:
In a project export, there will be a lists/
directory. Within the lists/
directory, there will be a subdirectory for each mailing list in the project.
Each exported mailing list contains a settings.ini
file generated from this template: https://github.com/socialplanning/opencore/blob/master/opencore/export/export_list_conf.ini.tmpl -- which should contain all data necessary to reconstruct a mailing list, including its mailto, list of managers, and workflow policies.
So, the import code will need to iterate through the subdirectories of lists/
and create a new mailing list object for each subdirectory, based on that subdirectory's settings.ini
data.
Remaining to do:
Ideally, we'd scan each project and determine if it has been modified at all since its last export. This looks like it will be very difficult though. Many objects don't cause the project' lastModifiedDate to change when the objects change (e.g. new mailing list messages, changes to team membership) and some objects evidently don't even update their OWN lastModifiedDate to reflect all changes (e.g. promoting a team member to admin)
I think a good enough solution would be to just check whether the project's wiki needs to be re-exported. This should be pretty easy, and also will give us the most speedup -- wiki export is the slowest part of project export by far. So the entire project would be exported into a new zipfile, but we would check if an existing wiki export is up-to-date and, if so, just move that wiki export into our new zipfile instead of re-exporting the wiki.
i.e. apply modifyportalcontent permission to site admin accounts
Include metadata upon reimport.
See e.g. http://openfsm.net/projects/ui18n/lists/ui18n-discussion/archive/2009/04/1240516617333/forum_view vs -- on the latest test-reimport site -- /projects/ui18n/lists/ui18n-discussion/archive/2009/04/1240516617000/forum_view
Allow admins to specify a mailing list to receive wiki change notices, or send (out of band) to project admins, or none.
Include this in project export and reimport.
After I implement a solution for socialplanning/opencore-listen#1 upstream in the Listen codebase, I need to merge that code to the opencore build profile and test it in my zopectl script.
We need to support reimport of wikis from a project export.
The structure of the project export means that data will be in two places.
The primary data will all live in the wiki_history/
subdirectory of the export. This is a BZR repository with full wiki history, preserving original commit dates, messages and authors. File contents are preserved in their "unbaked" wiki formats.
Meanwhile, file attachments will live in subdirectories of the pages/
subdirectory of the export. Historical revisions of file attachments are not preserved (by design). Current versions of attachments are organized by the pages that they are attached to. So attachments on the "project-home" page live in the pages/project-home/
subdirectory of the export.
Import can therefore be done in two phases:
pages
subdirectory, in any orderThe pages/
subdirectory also contains current wiki page contents, transformed into "baked" HTML with headers and necessary CSS links, so that the export's pages/
subdirectory is suitable for standalone drop-in publication to a web server. These should be discarded during import: they are lossy, and redundant with the lossless BZR history.
Allow admins to specify a mailing list that would receive project membership request emails, instead of sending it (out-of-band) to project admins.
Include this in project export and reimport.
This would be included in the join request email that's sent to project admins.
Allow admins to specify a wiki page that should receive a serialization of the project's team, automatically updated every time the team changes.
Need to think more about security here.
On the summary page, wiki pages end up being randomly ordered after reimport. The last modified dates are all correct on the pages themselves, but the summary page (and presumably the underlying RSS feed) isn't sorting by that somehow. Maybe we just need to reindex some catalog after import.
Need to investigate whether the existing Listen export functions handle message attachments. If not, we need to build that in to both export and import; in some cases it's essential content.
e.g. http://openfsm.net/projects/varal-da-violncia/summary
Need to create as admin instead, and then:
Make sure to preserve dates for all, and listed + role status for full fledged team members.
All of them are missing a lot of project-membership listings.
It seems like performing a reindex on membrane_tool's project_ids index fixes the @@team page, but not @@Profile. (I'm not sure about @@manage-team.)
I'm trying to reindex all indexes on portal_catalog now to see if that impacts @@Profile.
We'll need a script that exports all projects.
Here's some code that looks like it works, given constants PROJ_ID and OUTDIR:
>>> from opencore.utils import setup
>>> app=setup(app)
>>> from opencore.export.export_utils import *
>>> project = app.openplans.projects[PROJ_ID]
>>> status = get_status(project.id)
>>> status.features = ["wikipages", "mailinglists", "wikihistory"]
>>> outdir = OUTDIR
>>> tmpfd, tmpname = tempfile.mkstemp(suffix='.zip', prefix=TEMP_PREFIX, dir=outdir)
>>> tmp = os.fdopen(tmpfd, 'w')
>>> z = ZipFile(tmp, 'w')
>>> proj_dirname = badchars.sub('_', project.id)
>>> exporter = ContentExporter(project, None, status, proj_dirname, z)
>>> exporter.save()
>>> z.close()
>>> tmp.close()
Find them by code like:
for mem in app.openplans.portal_memberdata.keys():
if not IHandleMemberWorkflow(app.openplans.portal_memberdata[mem]).is_unconfirmed():
if IAnnotations(app.openplans.portal_memberdata[mem]).get("opencore.member.pending_requests", {}).items():
print mem, IAnnotations(app.openplans.portal_memberdata[mem]).get("opencore.member.pending_requests", {}).items()
Not sure what content there is other than "News" -- I'm pretty sure About/Contact/etc are all views instead of content, but we need to make sure we don't miss any.
We need to preserve:
for all attachments upon reimport. Right now dates are not being set properly upon reimport.
We have an export of user photos, but we don't currently reimport them.
When reimporting mailing lists, we need to make sure to include and fix up:
The listen moderation queues (both senders/subscribers, and messages) are, I believe, not currently exported or imported in any code.
Does anybody care?
Listen already has import features built in, but they aren't tested at all in an opencore setting.
The built-in export and import features pertain to:
The built in export/import features do not include metadata and settings on the list itself (including list managers) -- these need to be handled separately.
Add site manager views to list all unconfirmed members.
For each unconfirmed member, show:
For each unconfirmed member, allow manager to:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.