Comments (8)
I hope to have some news in Q2-Q3 on the topic...
from cvat.
Hi,
It is a feature in our mind which we don't have resources to implement till EOY for sure. If you can help it will be awesome. Development environment is described here: #18.
Several notes:
- Need to implement a separate Django application for the feature
- It isn't good to use turkic from VATIC because it isn't supported anymore. I would suggest using an official API (boto3?) - http://boto3.readthedocs.io/en/latest/reference/services/mturk.html
- Need to think how split a task on many HITs (by frame/image?) because HITs should be trivial. Also need to automatically create "verification" tasks to check results. In many case it will be necessary to merge annotation from several HITs.
- GUI to publish, view progress, ... Need to have sketch of it.
- The application should have REST API.
In general it is a big feature which cannot be easily implemented.
from cvat.
General requirements:
- Easy interface to create/review/accept an annotation task for MTurk (separate Django app).
- Probably need to have a separate instance on AWS for such tasks.
- Automatic authorization from MTurk, special group to access only links from MTurk django app
- Email notifications about deadlines to verify annotation tasks, etc...
Requirements (image annotation):
- One image per a segment. Thus it is easy to post such job as HIT.
- Several jobs for the same segment.
- Automatically correlate annotations in different jobs for the same segment. Automatically find "trash" in annotations and discard such HIT.
- Minimum number of jobs for a segment is 3.
- Maximum number of jobs for a segment is 7. After that need to verify or discard HITs due to low quality.
- One user cannot do annotations for the same segment (several similar jobs). Need to improve auth/permissions to access only specific jobs.
from cvat.
Hi, Wanted to know if any progress has been made in this direction :)
from cvat.
@nik13 , we are not working in the direction at the moment.
from cvat.
Nice overview of using a crowd sourcing platform (Toloka): https://www.youtube.com/watch?v=yyybisXHFc0
from cvat.
Has there been any progress on the integration with crowdsourcing platforms?
from cvat.
Very interested in this too!
from cvat.
Related Issues (20)
- AI tools won't appear on Windows 10 HOT 1
- Is there a way in CVAT Grafana Dashboard to view number of Images annotated by Annotators vs assigned HOT 1
- Last updated filter field does not work for tasks/projects/jobs HOT 7
- Cloud storage stopped working HOT 4
- Compatibility with Other Serverless Architectures
- 404 Accessing from outside the network using RemoteIT HOT 1
- Can't Move Projects Between Workspaces HOT 1
- Could not restore project backup HOT 11
- Exporting project with duplicate image names incorrect
- Uploading from S3, manifest.jsonl file needs to be in the same location as the image data
- Could not restore project backup HOT 3
- task.json not found error while uploading backup task HOT 1
- How to Provide SSL Certificate for Self Hosted CVAT application Single EC2 instance? HOT 1
- can't access property "length", e is undefined HOT 3
- Export Labels to Cloud Storage using API HOT 1
- YOLO format may export incorrectly if there are skeletons with repeated sublabels names
- How to change the location where uploaded images are saved HOT 1
- Access Error: 404 -- Not Found HOT 4
- wrong bbox height and width values on coco kp export for single keypoints HOT 2
- How to deploy custom YOLOv8 model HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cvat.