How Really Simple Licensing May Change Online Content Licensing

The Really Simple Licensing Collective (“RSL Collective”), a nonprofit dedicated to creating collective licensing solutions for content creators and publishers, has announced Really Simple Licensing (“RSL”), a new standard designed to stop crawlers from scraping websites for content without permission or compensation. If adopted, RSL could have major implications for both online platforms and the AI technologies that source content for training data from them.

The Current State of Licensing Online Content

One of the most pressing challenges facing both companies that publish content online and those who would like to license that content is that there is no effective and standardized licensing system for online content. As AI technologies emerge, this challenge has taken on a renewed urgency.

Currently, there is no way to track, enforce, or manage permissions for licensing online content. Nor is there a standard framework for price—opinions vary as to whether compensation for licensors should be based on frequency of use, content type, or other factors. This means that negotiations are done ad hoc, creating inefficiencies, uncertainty, and risk for both publishers trying to protect their work and the AI technologies that seek to use it.

What is RSL?

RSL allows publishers to define and automate licensing terms for their online content and is designed to ensure that AI companies pay online publishers when they use their content to build and train AI products.

In practice, RSL updates the traditional robots.txt protocol—a decades-old tool that websites use to tell automated bots, such as search engines, what pages they can or cannot crawl. While the traditional robots.txt protocol only allows for a simple “yes” or “no” instruction, RSL adds detailed, machine-readable licensing and royalty terms that specify how users are required to compensate the original publisher for using their content. For example, a publisher can set rules establishing that their content is free to use, or that a company must pay a fee each time the content is crawled, scanned or otherwise used.

Publishers can select a range of licensing, usage, and royalty models through RSL, including free, attribution, subscription, pay-per-crawl, and pay-per inference models, and the technology can be used to protect any digital content, including blog posts, news articles, images, videos, and audio. 

How RSL Could Improve the Online Content Licensing Landscape

RSL may simplify how publishers license their digital content. Instead of negotiating separate agreements with each entity seeking to use a publisher’s content (or having no agreement at all), publishers can use RSL to clearly state their licensing preferences in a machine-readable format. This could bring consistency and efficiency to a currently fragmented, time-consuming, and uncertain process. This may also have benefits to AI companies, who may want to license work for training without taking on the legal uncertainty and risk of an unsettled area of law.

What Websites and Online Publishers Should Know About RSL

• • • RSL Can be Adopted Now. Publishers can find common terms to add to their robots.txt files on the RSL website. Companies should consider what licensing model to use with RSL, taking into account the size of their audience, the type of online content they are publishing, and other business considerations.
    • RSL’s Effectiveness Remains to be Seen. Currently, RSL acts largely as a request and instruction system, meaning that online bots do not necessarily have to agree to the publisher’s requested licensing terms prior to accessing content. Additional partnership with other content delivery networks is needed to actually block or limit access to those who do not accept the publisher’s licensing terms.
    • Potential Legal Challenges. As for legal enforcement, major gray areas remain around licensing and usage for AI technologies. RSL may also raise other potential legal issues, such as the enforceability of RSL licensing agreements and antitrust risks from collective licensing. As RSL is tested in practice, we will monitor how AI companies respond (whether through negotiation, compliance, or attempts to bypass RSL), how the framework itself evolves to address any unforeseen issues, and any legal challenges that may arise in response.