Back to Label Studio

Compare Label Studio Editions

docs/source/guide/label_studio_compare.md

2.2.10-118.1 KB
Original Source

Label Studio is available to everyone as open source software (Label Studio Community Edition). There are also two paid editions: Starter Cloud and Enterprise.

<a class="Button" href="#Feature-comparison" target="_blank" style="margin-bottom: 2em;">See full feature comparison</a>

At-a-glance

<a class="Button" href="https://humansignal.com/" target="_blank" style="margin-bottom: 2em;">Learn about Enterprise</a>

Feature comparison

<table> <thead> <tr> <th>Functionality</th> <th>Community</th> <th>Starter Cloud</th> <th>Enterprise</th> </tr> </thead> <tr> <td colspan="4" style="text-align:center"><b>User Management</b></td> </tr> <tr> <td><b>Role-based workflows</b> <a href="https://docs.humansignal.com/guide/project_settings_lse#Annotation">Role-based automated workflows for annotators and reviewers.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">Limited</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Role-based access control</b> <a href="https://docs.humansignal.com/guide/admin_roles">Role-based access control into workspaces and projects: Admin, Manager, Reviewer, and Annotator.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Customizable permissions</b> <a href="https://docs.humansignal.com/guide/admin_permissions">Fine-grained control over which roles can execute specific actions, such as inviting members, managing cloud storage, deleting tasks, and more.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td colspan="4" style="text-align:center"><b>Data Management</b></td> </tr> <tr> <td><b>Data management view</b> <a href="manage_data.html">View and manage datasets and tasks in a project through the Data Manager view.</a></td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Multiple data formats</b> <a href="https://labelstud.io/playground/"> Label any data type from text, images, audio, video, time series data to multimodality.</a></td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Import data</b> <a href="tasks.html">Reference data stored in your database, cloud storage buckets, or local storage and label it in the browser.</a></td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Advanced cloud storage integrations</b> <a href="storage.html">Databricks Unity Catalog, IAM for AWS S3, Azure Blob Storage with Service Principal, and WIF for Google Cloud Storage connections.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Import pre-annotated data</b> <a href="predictions.html">Import pre-annotated data (predictions) into Label Studio for further refinement and assessment.</a></td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Export data</b> <a href="export.html">Export annotations as common formats like JSON, COCO, Pascal VOC and others.</a></td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Sync data</b> <a href="storage.html">Synchronize new and labeled data between projects and your external data storage.</a></td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Chat</b> <a href="/tags/chat.html">Generate and label conversation data. Interact live with an LLM of your choice.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">Limited</td> <td style="text-align:center">✅</td> </tr> <tr> <td colspan="4" style="text-align:center"><b>Project Management</b></td> </tr> <tr> <td><b>Organize data in projects</b> <a href="setup_project.html">Projects to manage data labeling activities.</a></td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Organize projects in workspaces</b> <a href="https://docs.humansignal.com/guide/workspaces">Organizes related projects by team, department, or product. Users can only access workspaces they associated with. </a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Personal sandbox workspace</b> <a href="https://docs.humansignal.com/guide/workspaces">Personal sandbox workspace for project testing and experimentation. </a></td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Templates</b> <a href="/templates">Templates to set up data labeling projects faster.</a></td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>AI assistant</b> <a href="https://docs.humansignal.com/guide/ask_ai">Use an LLM trained by HumanSignal to help you create and refine templates.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Project membership</b> <a href="https://docs.humansignal.com/guide/project_settings_lse#Members">Only users who are added as members to a project can view it. </a></td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Project-level roles</b> <a href="https://docs.humansignal.com/guide/project_settings_lse#Members">Annotator and Reviewer can be assigned to Annotator/Reviewer roles at a per-project level. </a></td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Project-level user settings</b> <a href="https://docs.humansignal.com/guide/project_settings_lse#Annotation">Multiple configuration options for how Annotators and Reviewers interact with tasks and what information they can see.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">Limited</td> <td style="text-align:center">✅</td> </tr> <tr> <td colspan="4" style="text-align:center"><b>Data Labeling Workflows</b></td> </tr> <tr> <td><b>Assign tasks</b> <a href="https://docs.humansignal.com/guide/manage_data#Assign-annotators-to-tasks">Assign tasks to certain annotators or reviewers.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Automatically assign tasks</b> <a href="https://docs.humansignal.com/guide/setup_project#Configure-high-impact-settings">Set rules and automate how tasks are distributed to annotators.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Simplified interface for Annotators</b> Annotator-specific labeling view that only shows assigned tasks.</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Bulk labeling</b> <a href="https://docs.humansignal.com/guide/labeling_bulk">Classify data in batches.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td colspan="4" style="text-align:center"><b>Customization & Development</b></td> </tr> <tr> <td><b>Programmable & embeddable interfaces</b> <a href="https://docs.humansignal.com/tags/reactcode">Build fully customized labeling and evaluation interfaces with React. Embed into your own applications.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Tag library</b> <a href="/tags">Use our tag library to customize the labeling interface by modifying pre-built templates or by building your own templates.</a></td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>White labeling</b> Use your company colors and logo to give your team a consistent experience. (Additional cost)</td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Plugins</b> <a href="https://docs.humansignal.com/guide/plugins">Use JavaScript to further enhance and customize your labeling interface.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>API/SDK & webhooks</b> <a href="api.html"> APIs, SDK, and webhooks for programmatically accessing and managing Label Studio.</a></td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td colspan="4" style="text-align:center"><b>Prompts</b></td> </tr> <tr> <td><b>Automated pre-labeling</b> <a href="https://docs.humansignal.com/guide/prompts_overview">Rapidly pre-label tasks using LLMs.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">Limited Trial</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>LLM fine-tuning and evaluation</b> <a href="https://docs.humansignal.com/guide/prompts_draft">Evaluate and fine-tune LLM prompts against a ground truth dataset.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">Limited Trial</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Run benchmarks</b> <a href="https://docs.humansignal.com/guide/prompts_draft#Evaluation-results">Compare model outputs against ground truth or rubric criteria.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">Limited Trial</td> <td style="text-align:center">✅</td> </tr> <tr> <td colspan="4" style="text-align:center"><b>Machine Learning</b></td> </tr> <tr> <td><b>Custom ML backends</b> <a href="ml.html">Connect a machine learning model to the backend of a project.</a></td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Active learning loops</b> <a href="https://docs.humansignal.com/guide/active_learning.html">Accelerate labeling with active learning loops.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Predictions from connected models</b> <a href="ml.html#Pre-annotations-predictions">Automatically label and sort tasks by prediction score with the ML model backend.</a></td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td colspan="4" style="text-align:center"><b>Analytics and Reporting</b></td> </tr> <tr> <td><b>Project dashboards</b> <a href="https://docs.humansignal.com/guide/dashboard_project">Dashboards for monitoring project progress.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Annotator performance dashboards</b> <a href="https://docs.humansignal.com/guide/dashboard_annotator">Dashboards to review and monitor individual annotator performance.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Activity logs</b> <a href="https://docs.humansignal.com/guide/admin_logs">Activity logs for auditing annotation activity by project.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Annotation history</b> View annotation history from the labeling interface.</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td colspan="4" style="text-align:center"><b>Quality Workflows</b></td> </tr> <tr> <td><b>Assign reviewers</b> <a href="https://docs.humansignal.com/guide/quality.html">Assign reviewers to review, fix and update annotations.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Automatic task reassignment</b> <a href="https://docs.humansignal.com/guide/quality.html">Reassign tasks with low agreement scores to new annotators.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Agreement metrics</b> <a href="https://docs.humansignal.com/guide/stats.html">Define how annotator consensus is calculated using pre-defined agreement metrics.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Custom agreement metrics</b> <a href="https://docs.humansignal.com/guide/custom_metric.html">Write your own custom agreement metric.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Comments and notifications</b> <a href="https://docs.humansignal.com/guide/comments_notifications.html">Team collaboration features like comments and notifications on annotation tasks.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Identify ground truths</b> <a href="https://docs.humansignal.com/guide/quality.html">Mark which annotations should be included in a Ground Truth dataset.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Overlap configuration</b> <a href="https://docs.humansignal.com/guide/project_settings_lse#Quality">Set how many annotators must label each sample.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Pause annotators</b> <a href="https://docs.humansignal.com/guide/quality#Pause-an-annotator">Pause an individual annotator's progress manually or based on pre-defined behaviors.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Annotation limits</b> <a href="https://docs.humansignal.com/guide/project_settings_lse#annotation-limit">Set a limit on how many annotations a user can submit within a project before their work is paused.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Annotator consensus matrices</b> <a href="https://docs.humansignal.com/guide/quality#Review-annotator-agreement-matrix">Matrices used to compare labeling results by different annotators.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Label distribution charts</b> <a href="https://docs.humansignal.com/guide/dashboard_project#Label-distribution">Identify possible problems with your dataset distribution, such as an unbalanced dataset.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td colspan="4" style="text-align:center"><b>Security and Support </b></td> </tr> <tr> <td><b>SSO</b> <a href="https://docs.humansignal.com/guide/auth_setup.html">Secure access and authentication of users via SAML SSO or LDAP.</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>SOC2</b> <a href ="https://heartex.com/security"> SOC2-compliant hosted cloud service or on-premise availability</a></td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Support portal</b> Access to a dedicated support portal.</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Uptime SLA</b> 99.9% uptime SLA</td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> <tr> <td><b>Customer Success Manager</b> Dedicated customer success manager to support onboarding, education, and escalations.</td> <td style="text-align:center">❌</td> <td style="text-align:center">❌</td> <td style="text-align:center">✅</td> </tr> </table>