vocadb / vocadb Goto Github PK

Currently, when writing a comment for an entry the user needs to switch to the discussion tab to view all comments. The user should be able to write comments on the first tab.

The main difficulty with this is synchronizing the two lists: should we only allow writing comments on the first tab, and if the user wants to delete that comment they would switch to the discussion tab? What about when they edit or delete the comment on the discussion tab, should it be updated on the first tab as well?

To avoid taking too much space, new comment box could somehow be minimized until the user starts typing.

It would be easier to translate if we are free to move the number/count within the string, for example "AlbumCount" (Views/Shared/MiscStrings) has "album(s)" in neutral/English, but it would be better if it was "{0} album(s)"

Most counting strings already have {0} parameters.

The one I noticed was:
Views/Shared/MiscStrings (AlbumCount, SongCount, ArtistCount) and
Views/Shared/EntryDetailStrings (Hits).

At least Japanese and English translations should be supported.

Open issues:

• Should tag names support other languages besides Japanese and English?
• Should tag names support any number of aliases?
• Should we allow creating a tag without any English characters?
• Should we keep all tag names unique?

The current API doesn't offer a way to retrieve lyrics for covers and remixes which on the website pull the data from the original version of the song.
Either the API behaviour should match the site behaviour for returning lyrics, or a way to determine the original song's ID should be added.

Currently active editor is updated when the edit page is opened. It would be possible to periodically refresh this timer as long as the page is open, so that it doesn't expire if the user is editing the entry for an extended period of time. We could also shorten the quiet period after which the entry is no longer assumed to be under editing, to avoid unnecessary edit warnings.

When viewing a playlist in "Details" mode, it would be nice if tags could be optionally displayed as well, similar to how it is done when searching for songs (there is an extra button with a tag icon on it to toggle display of tags)

For example, for the song Senbonzakura, we want to be able to list all utaite versions of that song that are on UtaiteDB. In this case the song includes a reference link to UtaiteDB and the same entry has been added there, but that's not always the case (most importantly the reverse is not always true: if the original is an utaite song, we generally don't add it on VocaDB).

This should be an on-demand feature that requires user interaction: a link that says for example "check for alternate versions on UtaiteDB", which either populates the list of alternate versions or shows a popup.

Because of the different conventions on VocaDB and UtaiteDB we have to support two different ways. If the song includes a reference link from VocaDB to UtaiteDB, we simply get derived versions of the song from UtaiteDB using the ID of the referred song. If the original isn't on UtaiteDB we have to search by link URL: find all non-original songs from UtaiteDB that include a reference link to this entry on VocaDB.

In this case we simply load derived versions of the song with ID 1052 on UtaiteDB. If there was no reference link to UtaiteDB, we would search UtaiteDB for songs with a reference link to this entry.

We need to make it easier for all the contributors to communicate, discuss guidelines, ask questions and such. This can be done by setting up a new forum based on something like discourse for example, or asking the forum to be created as a subforum somewhere else, such as reddit or VO.

However, it wouldn't be too much work to implement very lightweight discussions on VocaDB itself, working in the same style as the entry comments. A single level of subforums should be enough, under that posts and comments under posts. So, root -> subforums -> posts -> comments. Post editing should also be implemented, but this is even more simple.

For server-side processing of Markdown VocaDB uses MarkdownSharp, with some of its extra features turned on, roughly corresponding to the Github-flavored Markdown (GFM). Unfortunately the MarkdownSharp library hasn't been maintained in years and there's multiple libraries with much better performance available. Most of them don't support the same customizations that we're using though which is limiting our options to change. Switching to CommonMark could be an option as well once it's stabilized enough and there's a .NET parser for it.

List of extensions / deviations from the original Markdown spec:

• Automatic hyperlinks: no need to surround URLs with brackets (same as GFM)
• Strict bold italic: bolding and italics need non-word characters (such as whitespace or punctuation) on both sides (same as GFM).
• Automatic newlines (line breaks): every line break counts, no need to end the line with spaces or have multiple line breaks.

What we'd like to have additionally: option to disable images in comments (VocaDB doesn't really need these that often), or at least limit the maximum image size and file extension (to prevent animated gifs).

Some libraries to investigate:

• MarkdownDeep.
• Moonshine.
• CommonMark.NET.

TL;DR: It'd be good if we could replace MarkdownSharp library with something more efficient, but we must be able to keep the current customizations.

Album/artist additional images and tag/songlist main images are saved under the application folder, they should be moved under the static domain, like album/artist main thumbnails are already.

Code changes are mostly done, the actual migration work remains.

• album/artist additional images
• tag/songlist images

Coded messages like &&& become escaped and turn into &&& when sent through PM's.

The preview shows &&& so it's a little bit confusing when it turns out different.

I'm not sure if escaping is intended, but if it is maybe the preview should also show it escaped. It's such a minor thing but though I'd point it out. 🍡 Might be related or duplicate of # #38 or #39.

Maybe on the revision information page, you could enter a short message explaining what's wrong.

The discussion relative time stamp shows incorrect time (see image). The discussion preview in the "Basic Info" tab time stamps seem correct, so I don't think it's my local computer clock that's wrong.

Somewhat related, it would be nice if hovering over the relative time revealed the actual clock time (like Twitter)

User should be able to store a search configuration as a named preset. For example, all songs with the tag "vocarock" where the artist is "Kagamine Rin" and her child voicebanks. Song search presets would also function as dynamic song lists.

In addition to personal search configuration presets, there could be shared presets, in the same way as featured song lists. Only moderators are able to maintain these shared presets (the featured song list management role can probably be re-used).

Currently all messages are loaded at once, and notifications and received messages are split in the UI. The server should support loading messages for each folder individually, and also loading more messages than the initial amount.

Currently album/artist descriptions and song notes are generally written in English, which might put off Japanese/foreign users. It is possible to add translations support to these fields. It should be enough to have 2 fields: original/non-English description and English translation for now.

The artist entry should be editable by the verified user even if the entry is approved/locked and the user isn't a trusted user.

It's enough to allow editing the artist entry for now, later we could make the association give permission to edit all the songs and albums associated with that artist as well.

For songs such as http://vocadb.net/S/94746 where the original is an utaite song, the reference link could be converted to a link that looks the same as the normal "original version" link, with artist information and translated title.

If the current song is a derived version (not original), and the original isn't present on VocaDB, examine the list of external links. If there's one or more links to UtaiteDB, use the API to load information about those songs (most of the time there's only one). If any of the linked songs is an original version, show the linked song under the original version title and remove the UtaiteDB link from the external links list.

This could be done for derived versions as well: if the current song is an original version, call UtaiteDB to see if there are derived versions there and show them in the derived versions list, provided that they're not already there.

This would work on UtaiteDB as well in reverse: VocaDB links are converted to entry references.

It's also possible that the reference link doesn't directly point to the original version, but the original is still on the target site. Most of the time this would be because the same derived version has been added on both sites. This can be solved by also loading the original version of the referred song, although this won't work if the original isn't the immediate parent of the referred song.

At the moment, web.config is excluded from version control, the production web.config is updated by hand on the server. This is a problem mostly when libraries are updated, because those might require changes to the web.config, such as assembly redirects, and keeping track of those is difficult, especially because this needs to be done for both VocaDB and UtaiteDB.

All passwords need to be moved away from the main web.config, to external files. .NET supports loading Appsettings and connection strings from external files as well.

Additionally, the deploy script needs to be updated to run necessary transformations for production when the solution is compiled. Transformations need to be added for UtaiteDB as well.

when changing the number of results shown in the "All songs" tab in the artist's page, the songs aren't loaded unless you switch to another result's page

Mainly for uniformity, the PV player on the front page should use the playlist player which is used on the song list and song search pages already. This would probably allow us to remove some code as well.

We need to ensure that the playlist player is responsive enough for mobile devices. Additionally, I'd like to have the larger PV thumbnails and possibly artist string for each song row, like in the current player on the front page. This would be an option for the playlist player.

2015-06-06 01:50:51.0737 [] ERROR Unexpected database error - NHibernate.Exceptions.GenericADOException could not execute batch command.[SQL: SQL not available] Void DoExecuteBatch(System.Data.IDbCommand) at NHibernate.AdoNet.SqlClientBatchingBatcher.DoExecuteBatch(IDbCommand ps)
at NHibernate.AdoNet.AbstractBatcher.ExecuteBatch()
at NHibernate.AdoNet.AbstractBatcher.PrepareBatchCommand(CommandType type, SqlString sql, SqlType[] parameterTypes)
at NHibernate.Persister.Entity.AbstractEntityPersister.Delete(Object id, Object version, Int32 j, Object obj, SqlCommandInfo sql, ISessionImplementor session, Object[] loadedState)
at NHibernate.Persister.Entity.AbstractEntityPersister.Delete(Object id, Object version, Object obj, ISessionImplementor session)
at NHibernate.Action.EntityDeleteAction.Execute()
at NHibernate.Engine.ActionQueue.Execute(IExecutable executable)
at NHibernate.Engine.ActionQueue.ExecuteActions(IList list)
at NHibernate.Event.Default.AbstractFlushingEventListener.PerformExecutions(IEventSource session)
at NHibernate.Event.Default.DefaultFlushEventListener.OnFlush(FlushEvent event)
at NHibernate.Impl.SessionImpl.Flush()
at NHibernate.Transaction.AdoTransaction.Commit()
at VocaDb.Model.Service.Repositories.NHibernate.NHibernateRepository1.HandleTransaction(Action1 func, String failMsg) in c:\Sites\VocaDB\web-git\VocaDbModel\Service\Repositories\NHibernate\NHibernateRepository.cs:line 128
SqlException The DELETE statement conflicted with the REFERENCE constraint "FK_ActivityEntries_ArchivedTagVersions". The conflict occurred in database "VocaloidSite", table "dbo.ActivityEntries", column 'ArchivedTagVersion'.
The statement has been terminated. Void OnError(System.Data.SqlClient.SqlException, Boolean, System.Action`1[System.Action])

The Japanese translation of the user interface is out of date and incomplete. The edit interfaces aren't translated.

A large part of the userbase is Japanese, and having a complete and high quality translation in Japanese would probably make the site more accessible to them.

This involves more translation work than coding.

Probably similar to current Twitter authentication. Twitter authentication is implemented with DotNetOpenAuth, the code is almost directly from samples (I added some error handling). DotNetOpenAuth should probably be updated, making sure Twitter login still works.

Need to record user's language somehow, for example when logging in, check which culture is set. This could be used at least for entry report notifications.

Most views don't work very well in mobile at the moment. Font sizes are too small and things don't float properly. Top search bar is too wide as well.

I've been contemplating about removing the left side navigation, possibly moving the items to a second level navigation bar on the top and/or dropdown lists in the main navigation on the top.

The search view is the most important to get working properly.

Something has probably changed, all the embeds are broken at the moment.

For example: http://vocadb.net/S/81340

Similar to entry reports list, but for general non-urgent editing tasks. Anyone can view the list, all registered users can edit. Could be integrated with the entry reports functionality: one view for reports, another for other editing tasks.

Fields:

• Entry (album, artist, song, maybe events as well)
• Assigned user (who is primarily responsible for advancing the issue - anyone is able to assign themselves, moderators can assign other people)
• Status? (open, in progress, on hold, done) - "on hold" status might need explanation. After being marked as "done", a moderator can check and delete the task.
• Votes? (vote as more important)

Moderators should be able to delete any task with one click (+ confirmation). Done tasks don't need to be confirmed when deleting.

Suggesting tasks for adding entries that don't exist yet? For example "this album should be added". On the other hand, the user can just add a draft entry and then add that to this list.

Currently, when submitting a song, the song title and artists are filled automatically, but only for Nico PVs. This is because most of the time, the original PVs is on NicoNicoDouga, but obviously this isn't always the case. The original might be from SoundCloud or Youtube as well, and the same title auto-parsing could be used for those.

Only "original" PVs should be considered, reprints are never parsed.

Current status

Currently we're using ASP.NET bundling and minification library. There's separate bundles for all main views, in addition to a couple of shared bundles. As the number of JavaScript files grows, managing dependencies is increasingly becoming a problem. We don't want to include all JavaScript files on all pages, and the files need to be included in the right order, so managing the bundles becomes complicated.

Goal

Nowadays TypeScript and EcmaScript (JavaScript) include built-in support for modules. Dependencies can be defined by importing modules.

When all dependencies are defined with import/export, a module bundling tool like Webpack can be used to create the bundles.

Of course this would be nearly impossible to do at once, so if we are to consider using Webpack, it has to be done somehow incrementally per file. Knockout includes some support for dynamically loading modules, but since VocaDB isn't a SPA (except for maybe the search page), it would be ok to include bundles per ASP.NET page.

Ideally we'd have one big bundle for common dependencies, plus page-specific bundles for page's viewmodels, similar to current ASP.NET bundles. For example, "bundles/main", which includes files used on most pages, "bundles/song/edit", which includes files on song/edit page (SongEditViewModel.ts + /song/edit.ts + their dependencies).

It's more clear to have ratings separate from album collection. Additionally, optional review text could be added. These reviews and ratings could be displayed similar to comments, perhaps on a separate tab. Review date could also be displayed. It would probably be useful to set review language.

Generally it doesn't make sense to specify both. Some kind of warning should be displayed. Alternatively, this could be prevented altogether. Guidance might need to be updated.

For example, there are multiple tags for MMD, which in turn is grouped under animation. Animation should be the root category, then MMD under it.

Open questions:

1. Presentation: should the root tags page display full hierarchies?
2. Can user assign tags from all levels of the hierarchy (see below for example)?
3. Does one tag need to be able to appear in multiple places, under different parents?

It would be possible to implement hierarchies using either tags or tag categories. The main difference is that the categories cannot be used as tags themselves. Not sure if this is needed or not. The root category is usually something too abstract like "genres", but all levels below that could potentially be used as tags. For example, for animation it could be:

• Animation (category)
  • MMD (tag/category)
    • MMD_Cup (category?)
      • MMD Cup 11 (tag)
    • MMD_models (category?)
      • Lat_Miku (tag)

Not sure if "MMD Cup" itself is useful as a tag. It could be a category, or skipped altogether.

Also, consider tagging of MMD models. The hierarchy could be designed in multiple different ways.

• Animation -> MMD -> MMD models -> Lat -> Lat Miku
• Animation -> Models -> Lat Miku
• Animation -> Models -> MMD models -> Miku -> Lat Miku.

One logical way of implementing this would be to have categories act as "folders" or "groups" for tags and subcategories. A tag could appear in multiple categories/folders/groups.

This doesn't really work for genres though. For example, consider

• Genres -> electronic -> trance -> euphoric trance

Here, electronic, trance and euphoric trance should all be usable as tags. And when browsing the "electronic" tag, the user would probably want to see all entries tagged with the trance and euphoric trance tags as well.

The server should limit the rate of comments (and maybe private messages) being posted by a single user, for example one per second. This is mostly to prevent spam.

• Does not affect trusted users (or moderators).
• Can be implemented as artificial delay by the server.
• Concurrent requests are not allowed.
• At first we can implement this only for "limited" users.

It would be useful to provide functionality for associating UTAU voicebanks with voice providers, illustrators and other related people. The groups field could be repurposed for this, but it would have to be renamed and/or a descriptive role attribute (indicating whether the artist is a circle, voice provider or illustrator) would need to be added.

Not sure whether it should be a warning or error. If it's considered an error, a unique database index should be added to ensure database integrity. We need to clear up the existing duplicates first though.

The only use case that I know of for adding the same artist multiple times is when the artist has multiple roles and some of them should be marked as support, while others are not. For example, if the artist provides backing vocals for a song they composed, they should be support for vocals, but main for composing role. These are very rare though, there's only a handful of cases in the database. We could still allow adding the artist twice if one instance is support and the other is not support, and disallow adding the same artist multiple times in all other cases.

It's frequent to find confusing Artist's/producer's names and collisions;
Also, it's not unusual to find that editors adding the wrong artist just because they have a similar name.
But crediting artists by their mylist number is consistent on NND, so we migh as well use it to make adding artists to songs and albums less error prone and faster.

By spec, all HTML is allowed by Markdown. Currently we're HTML encoding all text before it is passed to the Markdown parser. This prevents the most obvious XSS attacks, but not all. It'd be better to sanitize the generated HTML with a whitelist of allowed tags. Obviously we can't just HTML encode all of the HTML generated by the Markdown parser (or strip HTML tags), because otherwise using Markdown would be pointless to begin with.

The HtmlSanitizer library could possibly be used for this. There's also the Web Protection Library by Microsoft, but I've heard it's not very good. CsQuery HTML parsing library could also be used.

Currently titles can only be translated to 4 "content language selection" presets: Non-English, Romanized, English and Undefined (used for aliases). There's demand to support translations into other languages as well, for example German and Russian, or a Japanese title could be translated into Chinese.

First step is to allow specifying the exact language for the "non-English" name. This could be done similar to lyrics: there's fixed fields for original name and Romanization, plus variable number of fields for translations. For entry titles, aliases should be kept separate from translations of the primary name.

I think it's best to start with mapping those new language options to the current content language selection. This would still allow only English translation to be sorted and displayed in listings, but it'd be an improvement compared to current state, and would prepare the way for further improvements.

• Edit view: change current primary name editor to similar to lyrics. Original name, Romanized name, translations. Language can be selected (from dropdown) for original name and translations. Language options are identical to lyrics, including other/unknown.
  • Editing for aliases will remain as it is.
• Song page: all translations and aliases, including original and Romanized will be shown. First name to be shown is chosen like it's done now.
• Translations other than English will be treated like aliases for now. They cannot be used for sorting or as display name.

TODO: is there need to specify translations for aliases? This could be used to hide/show translations of aliases based on user's language setting. It would also enable varying artist name (with translations) per album/song (see VocaDB/community#68).

VocaDB is still using the default ASP.NET request validation which prevents posting HTML or other suspicious inputs.

However, this mechanism is too limited, as many entries use characters that are currently blocked by the request validation. A better way would be to disable request validation at least for entry submissions and instead make sure that output is encoded in all cases. Output encoding should obviously be done in any case, even if request validation is still enabled.

This task includes mostly testing, making sure that cross site scripting isn't possible.

Many artists, especially western artists share full songs on bandcamp.

Songs (and albums) from bandcamp can be embedded, but the embed code requires song ID. Bandcamp had an API for retrieving ID from song URL, but they closed that, so probably the only possibility is some ugly HTML parsing similar to piapro, or requiring the user to paste bandcamp embed code instead of song URL. Would also need to make sure that artist information and length are parsed, if possible.

Otherwise there is nothing preventing us from implementing this.