[ClusterLabs] [EXT] Login is required for now on projects.clusterlabs.org

[Jan Friesse]

On 24/11/2025 15:04, Windl, Ulrich wrote:
> Hi!
> 
> I think I would have tried to return a "please crawl elsewhere!" message to these crawlers instead.

... or do what (sadly) many other sites are doing these days - implement 
https://github.com/TecharoHQ/anubis.

Honza


> What if the crawlers use an account now?
> 
> Kind regards,
> Ulrich Windl
> 
>> -----Original Message-----
>> From: Users <users-bounces at clusterlabs.org> On Behalf Of Reid Wahl
>> Sent: Tuesday, November 18, 2025 1:51 AM
>> To: Cluster Labs - All topics related to open-source clustering welcomed
>> <users at clusterlabs.org>
>> Subject: [EXT] [ClusterLabs] Login is required for now on
>> projects.clusterlabs.org
>>
>>
>> A while back, we made tasks, commits, and source files on
>> projects.clusterlabs.org publicly accessible by default.
>>
>> Lately we've been inundated with HTTP requests from AI crawler bots
>> (from many IP addresses) that ignore robots.txt. To mitigate the
>> effects, we've set the permissions to "All Users" for the apps that
>> access tasks, commits, and source files. This means that no matter
>> what the permissions are for any particular object, you are now
>> required to log in before you can view it.
>>
>> It's easy to create an account, and you don't need any additional
>> permissions to view most things once you're logged in.
>>
>> We will probably continue requiring a login to view commits and source
>> code. I can't think of any reason those need to be publicly visible on
>> projects.clusterlabs.org, since they're already visible on GitHub.
>>
>> We hope to make the tasks/bug reports visible without a login again in
>> the future.
>>
>> --
>> Regards,
>>
>> Reid Wahl (He/Him)
>> Senior Software Engineer, Red Hat
>> RHEL High Availability - Pacemaker
>>
>> _______________________________________________
>> Manage your subscription:
>> https://lists.clusterlabs.org/mailman/listinfo/users
>>
>> ClusterLabs home: https://www.clusterlabs.org/
> _______________________________________________
> Manage your subscription:
> https://lists.clusterlabs.org/mailman/listinfo/users
> 
> ClusterLabs home: https://www.clusterlabs.org/
>