Distributed Press resilience challenge: restoring Desarquivo

CHALLENGE

The internet, an essential space for preserving and sharing knowledge, is becoming increasingly fragile. Studies estimate that over 30% of links on the web become inaccessible within a decade, with countless valuable resources disappearing due to server failures, outdated platforms, and lack of maintenance.

One such case was Desarquivo: an extensive digital archive documenting feminist, anarchist, and anti-racist activism in Brazil since 2011. Developed on an outdated Drupal platform and hosted on a shared server that had reached full capacity, Desarquivo had become unstable, putting its valuable collection of more than 1,500 documents at risk.

SOLUTION

With the Filecoin Foundation for the Decentralized Web-supported Resilience Grant, Distributed Press aims to raise awareness about the fragility of digital resources in centralized infrastructures, and present alternatives on decentralized, resilient web foundations such as peer-to-peer protocols.
We invited organizations with lost assets to apply to this opportunity and analyzed the submissions using technical criteria. The two selected organizations were offered the chance to have their lost content recovered and moved onto a static site, and then published using HTTPS, IPFS and Hypercore protocols. This publishing approach eliminates reliance on a single centralized server for them, ensuring long-term accessibility and stability.

RESULT

Imotirõ, the collective behind Desarquivo, applied and was selected for the Resilience Grant as they complied with our technical and organizational criteria. By leveraging decentralized technologies and the use of static-sites, we recovered and distributed the website Desarquivo, safeguarding it from digital obsolescence, and preserving its historical and cultural records for future generations. The new, more resilient infrastructure enables the archive to thrive without being constrained by outdated software or limited server capacity.

Source: https://desarquivo.org/node/1467/

FULL STORY

The challenge of digital resilience

The internet is an essential space for preserving and sharing knowledge, but it is also fragile. Link rot and single points of failure threaten the longevity of digital archives. A study by Pew Research Center found that this happens even to government webpages, with 20% of them containing at least one broken link.

To address this issue, Distributed Press launched the Resilience Grant, an initiative designed to empower organizations working with crucial digital archives by transitioning them to decentralized, resilient web infrastructures. Within the Resilience Grant, we intended to raise awareness about the fragility of digital resources in centralized infrastructures, and present alternatives to be found on decentralized, resilient web foundations.

The selected organization: Imotirõ and Desarquivo project

Imotirõ is a cultural association of researchers, educators, activists, and artists working on interdisciplinary knowledge production. They applied to the Resilient Grant with the hope of recovering one of their most significant projects, Desarquivo. Desarquivo is an extensive digital archive documenting feminist, anarchist, and anti-racist activism in Brazil since 2011. It held more than 1,500 documents, including texts, artistic practices, and self-organized political movements, dating back to the late 1990s. However, due to technical and financial constraints, the project had become increasingly unstable and was at risk of disappearing.

When receiving their submission to the Resilience Grant, we decided that the case complied with our technical and organizational selection criteria. Technically, they had sufficient backup of their lost content. Organizationally, there's alignment since they are a purpose-driven organization that creates positive social impact for collectives facing systemic inequity and prejudice, while adhering to open knowledge ethics.

At risk of dissappearing

Desarquivo was initially developed on Drupal, a powerful but resource-intensive content management system. Over time, the platform became outdated, making it difficult to maintain: the version of Drupal used in Desarquivo stopped being maintained by the development community in 2016. The instability of the Drupal-based site and the diminishing support for its version in free software communities posed a significant risk to the archive’s accessibility. Additionally, the archive was hosted on Njira, a shared server that had reached full capacity, preventing further updates and maintenance.

The recovery

When tasked with recovering Desarquivo’s website, we had full access to its code, archives, and database, which was crucial for restoring the site to its original state. Additionally, the team provided Docker configurations, allowing us to replicate the exact environment locally for recovery.

The initial restoration yielded an 8GB site with over 2,000 files, revealing significant redundancy in its dataset. We identified duplicate records and inefficient pagination inflating the archive's size. After consulting with Desarquivo, we proceeded to optimize pagination, reducing the total footprint from 8GB to just 2GB without compromising content integrity.

Further optimizations included:

  • Deprecation cleanup: removing an obsolete login functionality.

  • Search integration: replacing the internal search with DuckDuckGo, eliminating the need to maintain a dedicated search index.

Next, we leveraged our own tools to detect link rot across the site’s hypermedia. We found that nearly 200 external links were misconfigured due to faulty validation in Drupal, preventing them from resolving correctly. After fixing these, all outbound links now direct users to their intended destinations.

Finally, we converted the Drupal-based content into a static site, significantly reducing server load, enhancing stability, security and longevity. The recovered website was published via regular (HTTPS) and distributed web: IPFS and Hypercore protocols rely on peer-to-peer networks. By decentralizing storage, Desarquivo no longer relies on a single server, eliminating a critical point of failure and securing the archive’s availability for years to come. Curious about the role of distributed protocols? Read more in the Appendix.

And it's back! Readers now have multiple options to access Desarquivo:

A step toward a more resilient web

In these times of unprecedented attacks on human rights defenders, diversity and equity advocates, we need to commit to the longevity and safety of critical historical and cultural records. As social threats add to the already too common technical constraints, digital resilience becomes increasingly urgent. We feel this initiative can set a precedent for other collectives, researchers, and archivists looking to future-proof their work.
Learn more about Desarquivo and Imotirõ’s mission, and how Distributed Press continues to foster a more resilient web.

Appendix: Why go distributed?

We believe that distributed web (Dweb) and the use of decentralized technologies like the InterPlanetary File System (IPFS) and Hypercore offer robust solutions to tackle linkrot and degradation of digital assets by ensuring content remains both accessible and verifiable.

IPFS and content addressing

IPFS addresses link rot by utilizing content-addressable storage, where each file is identified by a unique cryptographic hash, known as a Content Identifier (CID). This system ensures that content can be retrieved from any node storing the data, independent of its original location, thereby eliminating single points of failure. (Source: https://starlinglab.org)

Hypercore and versioned data

Hypercore, a foundational component of the Dat Protocol, offers a distributed, append-only log structure that maintains a complete history of data changes. This versioning capability allows users to access specific data versions, mitigating issues of content drift and ensuring the integrity of digital archives. (Source: https://www.datprotocol.com/)

Post by